Enhancing accuracy and interpretability of machine learning models using super learning and permutation feature importance techniques in digital soil mapping 

2021 ◽  
Author(s):  
Ruhollah Taghizadeh-Mehrjardi ◽  
Nikou Hamzehpour ◽  
Maryam Hassanzadeh ◽  
Karsten Schmidt ◽  
Thomas Scholten
Keyword(s):  
Machine Learning ◽  
Learning Strategy ◽  
Soil Characteristics ◽  
Soil Mapping ◽  
Digital Soil Mapping ◽  
Learning Models ◽  
Wide Range ◽  
Feature Importance ◽  
Base Learner ◽  
Machine Learning Models

<p>The digital soil mapping (DSM) approach predicts soil characteristics based on the relationship between soil observations and related covariates using machine learning (ML) models. In this research, we applied a wide range of machine learning models (12 base learners) to predict and map soil characteristics. To enhance accuracy and interpretability we combined the base learner predictions using super learning strategy. However, a major problem of using super learning and complex models is that the explicit share of individual covariates persons in the overall result cannot be explicitly quantified. To overcome this restriction and make the super learning models interpretable, we employed model-agnostic interpretation tools, for example, permutation feature importance. Particularly, we integrated the weight assigned to each ML base learner obtained by super learning and the ranked ML base learner’s covariates obtained by permutation feature importance to explore the contribution of covariates on the final prediction. We tested our super learning and permutation feature importance techniques to predict and mapping physicochemical soil characteristics of Urmia Playa Lake (UPL) sediments in Iran. As expected, our results indicated that super leaning could significantly improve the ML accuracies for predicting soil characteristics of single base learners. In terms of root mean square error, super learning improved over the performance of the linear regression by an average of 45.7%. Furthermore, the permutation feature importance allowed us to interpret our results better and prove the significant contribution of geomorphological features and groundwater data in predicting soil characteristics of UPL sediments.</p>

Enhancing the accuracy of machine learning models using the super learner technique in digital soil mapping

Geoderma ◽  
2021 ◽  
Vol 399 ◽  
pp. 115108
Author(s):  
Ruhollah Taghizadeh-Mehrjardi ◽  
Nikou Hamzehpour ◽  
Maryam Hassanzadeh ◽  
Brandon Heung ◽  
Maryam Ghebleh Goydaragh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Soil Mapping ◽  
Digital Soil Mapping ◽  
Learning Models ◽  
Super Learner ◽  
Machine Learning Models

scholarly journals AB0652 MACHINE LEARNING TO PREDICT EARLY TNF INHIBITOR USERS IN PATIENTS WITH ANKYLOSING SPONDYLITIS

2020 ◽  
Vol 79 (Suppl 1) ◽  
pp. 1620.1-1621
Author(s):  
J. Lee ◽  
H. Kim ◽  
S. Y. Kang ◽  
S. Lee ◽  
Y. H. Eun ◽  
...  
Keyword(s):  
Machine Learning ◽  
Ankylosing Spondylitis ◽  
Tnf Inhibitors ◽  
Tnf Inhibitor ◽  
Ann Model ◽  
Learning Models ◽  
Feature Importance ◽  
Importance Analysis ◽  
Baseline Characteristics ◽  
Machine Learning Models

Background:Tumor necrosis factor (TNF) inhibitors are important drugs in treating patients with ankylosing spondylitis (AS). However, they are not used as a first-line treatment for AS. There is an insufficient treatment response to the first-line treatment, non-steroidal anti-inflammatory drugs (NSAIDs), in over 40% of patients. If we can predict who will need TNF inhibitors at an earlier phase, adequate treatment can be provided at an appropriate time and potential damages can be avoided. There is no precise predictive model at present. Recently, various machine learning methods show great performances in predictions using clinical data.Objectives:We aim to generate an artificial neural network (ANN) model to predict early TNF inhibitor users in patients with ankylosing spondylitis.Methods:The baseline demographic and laboratory data of patients who visited Samsung Medical Center rheumatology clinic from Dec. 2003 to Sep. 2018 were analyzed. Patients were divided into two groups: early TNF inhibitor users treated by TNF inhibitors within six months of their follow-up (early-TNF users), and the others (non-early-TNF users). Machine learning models were formulated to predict the early-TNF users using the baseline data. Additionally, feature importance analysis was performed to delineate significant baseline characteristics.Results:The numbers of early-TNF and non-early-TNF users were 90 and 509, respectively. The best performing ANN model utilized 3 hidden layers with 50 hidden nodes each; its performance (area under curve (AUC) = 0.75) was superior to logistic regression model, support vector machine, and random forest model (AUC = 0.72, 0.65, and 0.71, respectively) in predicting early-TNF users. Feature importance analysis revealed erythrocyte sedimentation rate (ESR), C-reactive protein (CRP), and height as the top significant baseline characteristics for predicting early-TNF users. Among these characteristics, height was revealed by machine learning models but not by conventional statistical techniques.Conclusion:Our model displayed superior performance in predicting early TNF users compared with logistic regression and other machine learning models. Machine learning can be a vital tool in predicting treatment response in various rheumatologic diseases.Disclosure of Interests:None declared

scholarly journals Machine learning to predict early TNF inhibitor users in patients with ankylosing spondylitis

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Seulkee Lee ◽  
Yeonghee Eun ◽  
Hyungjin Kim ◽  
Hoon-Suk Cha ◽  
Eun-Mi Koh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Logistic Regression ◽  
Ankylosing Spondylitis ◽  
Tnf Inhibitor ◽  
Ann Model ◽  
Learning Models ◽  
Feature Importance ◽  
Importance Analysis ◽  
Baseline Characteristics ◽  
Machine Learning Models

AbstractWe aim to generate an artificial neural network (ANN) model to predict early TNF inhibitor users in patients with ankylosing spondylitis. The baseline demographic and laboratory data of patients who visited Samsung Medical Center rheumatology clinic from Dec. 2003 to Sep. 2018 were analyzed. Patients were divided into two groups: early-TNF and non-early-TNF users. Machine learning models were formulated to predict the early-TNF users using the baseline data. Feature importance analysis was performed to delineate significant baseline characteristics. The numbers of early-TNF and non-early-TNF users were 90 and 505, respectively. The performance of the ANN model, based on the area under curve (AUC) for a receiver operating characteristic curve (ROC) of 0.783, was superior to logistic regression, support vector machine, random forest, and XGBoost models (for an ROC curve of 0.719, 0.699, 0.761, and 0.713, respectively) in predicting early-TNF users. Feature importance analysis revealed CRP and ESR as the top significant baseline characteristics for predicting early-TNF users. Our model displayed superior performance in predicting early-TNF users compared with logistic regression and other machine learning models. Machine learning can be a vital tool in predicting treatment response in various rheumatologic diseases.

scholarly journals Predictive Modeling for Frailty Conditions in Elderly People: Machine Learning Approaches (Preprint)

2019 ◽  
Author(s):  
Adane Tarekegn ◽  
Fulvio Ricceri ◽  
Giuseppe Costa ◽  
Elisa Ferracin ◽  
Mario Giacobini
Keyword(s):  
Machine Learning ◽  
Older Adults ◽  
Elderly People ◽  
Emergency Admission ◽  
Evaluation Metrics ◽  
Support Vector ◽  
Learning Models ◽  
Increased Risk ◽  
Wide Range ◽  
Machine Learning Models

BACKGROUND Frailty is one of the most critical age-related conditions in older adults. It is often recognized as a syndrome of physiological decline in late life, characterized by a marked vulnerability to adverse health outcomes. A clear operational definition of frailty, however, has not been agreed so far. There is a wide range of studies on the detection of frailty and their association with mortality. Several of these studies have focused on the possible risk factors associated with frailty in the elderly population while predicting who will be at increased risk of frailty is still overlooked in clinical settings. OBJECTIVE The objective of our study was to develop predictive models for frailty conditions in older people using different machine learning methods based on a database of clinical characteristics and socioeconomic factors. METHODS An administrative health database containing 1,095,612 elderly people aged 65 or older with 58 input variables and 6 output variables was used. We first identify and define six problems/outputs as surrogates of frailty. We then resolve the imbalanced nature of the data through resampling process and a comparative study between the different machine learning (ML) algorithms – Artificial neural network (ANN), Genetic programming (GP), Support vector machines (SVM), Random Forest (RF), Logistic regression (LR) and Decision tree (DT) – was carried out. The performance of each model was evaluated using a separate unseen dataset. RESULTS Predicting mortality outcome has shown higher performance with ANN (TPR 0.81, TNR 0.76, accuracy 0.78, F1-score 0.79) and SVM (TPR 0.77, TNR 0.80, accuracy 0.79, F1-score 0.78) than predicting the other outcomes. On average, over the six problems, the DT classifier has shown the lowest accuracy, while other models (GP, LR, RF, ANN, and SVM) performed better. All models have shown lower accuracy in predicting an event of an emergency admission with red code than predicting fracture and disability. In predicting urgent hospitalization, only SVM achieved better performance (TPR 0.75, TNR 0.77, accuracy 0.73, F1-score 0.76) with the 10-fold cross validation compared with other models in all evaluation metrics. CONCLUSIONS We developed machine learning models for predicting frailty conditions (mortality, urgent hospitalization, disability, fracture, and emergency admission). The results show that the prediction performance of machine learning models significantly varies from problem to problem in terms of different evaluation metrics. Through further improvement, the model that performs better can be used as a base for developing decision-support tools to improve early identification and prediction of frail older adults.

scholarly journals Bi-LSTM Sentiment Classifier for Climate Change Issues in South Korea

2019 ◽  
Vol 8 (2S6) ◽  
pp. 295-299
Keyword(s):  
Climate Change ◽  
Machine Learning ◽  
Big Data ◽  
South Korea ◽  
Sentiment Analysis ◽  
Training Data ◽  
Learning Models ◽  
Wide Range ◽  
Machine Learning Models ◽  
Big Data Technology

A sentiment analysis using SNS data can confirm various people’s thoughts. Thus an analysis using SNS can predict social problems and more accurately identify the complex causes of the problem. In addition, big data technology can identify SNS information that is generated in real time, allowing a wide range of people’s opinions to be understood without losing time. It can supplement traditional opinion surveys. The incumbent government mainly uses SNS to promote its policies. However, measures are needed to actively reflect SNS in the process of carrying out the policy. Therefore this paper developed a sentiment classifier that can identify public feelings on SNS about climate change. To that end, based on a dictionary formulated on the theme of climate change, we collected climate change SNS data for learning and tagged seven sentiments. Using training data, the sentiment classifier models were developed using machine learning models. The analysis showed that the Bi-LSTM model had the best performance than shallow models. It showed the highest accuracy (85.10%) in the seven sentiments classified, outperforming traditional machine learning (Naive Bayes and SVM) by approximately 34.53%p, and 7.14%p respectively. These findings substantiate the applicability of the proposed Bi-LSTM-based sentiment classifier to the analysis of sentiments relevant to diverse climate change issues.

Building High Performance Explainable Machine Learning Models for Social Media-based Substance Use Prediction

2020 ◽  
Vol 29 (03n04) ◽  
pp. 2060009
Author(s):  
Tao Ding ◽  
Fatema Hasan ◽  
Warren K. Bickel ◽  
Shimei Pan
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Substance Use ◽  
Human Behavior ◽  
High Performance ◽  
Supervised Machine Learning ◽  
Learning Models ◽  
Wide Range ◽  
And Behavior ◽  
Machine Learning Models

Social media contain rich information that can be used to help understand human mind and behavior. Social media data, however, are mostly unstructured (e.g., text and image) and a large number of features may be needed to represent them (e.g., we may need millions of unigrams to represent social media texts). Moreover, accurately assessing human behavior is often difficult (e.g., assessing addiction may require medical diagnosis). As a result, the ground truth data needed to train a supervised human behavior model are often difficult to obtain at a large scale. To avoid overfitting, many state-of-the-art behavior models employ sophisticated unsupervised or self-supervised machine learning methods to leverage a large amount of unsupervised data for both feature learning and dimension reduction. Unfortunately, despite their high performance, these advanced machine learning models often rely on latent features that are hard to explain. Since understanding the knowledge captured in these models is important to behavior scientists and public health providers, we explore new methods to build machine learning models that are not only accurate but also interpretable. We evaluate the effectiveness of the proposed methods in predicting Substance Use Disorders (SUD). We believe the methods we proposed are general and applicable to a wide range of data-driven human trait and behavior analysis applications.

scholarly journals Permutation-based identification of important biomarkers for complex diseases via machine learning models

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Xinlei Mi ◽  
Baiming Zou ◽  
Fei Zou ◽  
Jianhua Hu
Keyword(s):  
Machine Learning ◽  
Human Disease ◽  
Molecular Mechanisms ◽  
The Cancer Genome Atlas ◽  
Support Vector ◽  
Individual Feature ◽  
Learning Models ◽  
Efficient Manner ◽  
Feature Importance ◽  
Machine Learning Models

AbstractStudy of human disease remains challenging due to convoluted disease etiologies and complex molecular mechanisms at genetic, genomic, and proteomic levels. Many machine learning-based methods have been developed and widely used to alleviate some analytic challenges in complex human disease studies. While enjoying the modeling flexibility and robustness, these model frameworks suffer from non-transparency and difficulty in interpreting each individual feature due to their sophisticated algorithms. However, identifying important biomarkers is a critical pursuit towards assisting researchers to establish novel hypotheses regarding prevention, diagnosis and treatment of complex human diseases. Herein, we propose a Permutation-based Feature Importance Test (PermFIT) for estimating and testing the feature importance, and for assisting interpretation of individual feature in complex frameworks, including deep neural networks, random forests, and support vector machines. PermFIT (available at https://github.com/SkadiEye/deepTL) is implemented in a computationally efficient manner, without model refitting. We conduct extensive numerical studies under various scenarios, and show that PermFIT not only yields valid statistical inference, but also improves the prediction accuracy of machine learning models. With the application to the Cancer Genome Atlas kidney tumor data and the HITChip atlas data, PermFIT demonstrates its practical usage in identifying important biomarkers and boosting model prediction performance.

scholarly journals Design and Manufacture of a Multiband Rectangular Spiral-Shaped Microstrip Antenna Using EM-Driven and Machine Learning

2021 ◽  
Vol 27 (1) ◽  
pp. 29-40
Author(s):  
Ashrf Aoad
Keyword(s):  
Machine Learning ◽  
Microstrip Antenna ◽  
Machine Learning Algorithms ◽  
Mobile Systems ◽  
Training Data ◽  
Learning Models ◽  
Prediction Ability ◽  
Antenna Structure ◽  
Wide Range ◽  
Machine Learning Models

This paper presents a multiband rectangular microstrip antenna using spiral-shaped configurations. The antenna has been designed by combining two configurations of microstrip and spiral with consideration of careful selection of the substrate material, the dimension of the rectangular microstrip, the distance between the turned spiral, and the number of turns of the spiral. The efficiency and accuracy have been improved using machine learning algorithms as well. Machine learning has been studied to model the proposed antenna based on the performance requirements, which requires a sufficient training data to improve the accuracy. Three different machine learning models are applied to improve the accuracy and generalization performance and compared to simulation and measurement results. Simulation, measurement, and machine learning results confirm that the proposed antenna is a new electrically small and operating over a wide range of high-frequency bands between 1 GHz–4 GHz. Machine learning models have the best prediction ability with a mean square error (MSE) of 0.03, and 0.05. The antenna structure and size are compatible and suitable for several multi-band wireless mobile systems operating in L-band and S-band. The results, such as directivity, Half-Power Beamwidth, Voltage Standing Wave Ratio (VSWR), and S-parameter curves, are analysed and compared with the numerical formulation for both spiral and microstrip antennas.

A systematic exploration of ΔΔG cutoff ranges in machine learning models for protein mutation stability prediction

2018 ◽  
Vol 16 (05) ◽  
pp. 1840022 ◽  
Author(s):  
Richard Olney ◽  
Aaron Tuor ◽  
Filip Jagodzinski ◽  
Brian Hutchinson
Keyword(s):  
Machine Learning ◽  
Amino Acid ◽  
Amino Acid Substitutions ◽  
Learning Models ◽  
Systematic Exploration ◽  
Wide Range ◽  
The Stability ◽  
Experimental Values ◽  
Machine Learning Models

Discerning how a mutation affects the stability of a protein is central to the study of a wide range of diseases. Mutagenesis experiments on physical proteins provide precise insights about the effects of amino acid substitutions, but such studies are time and cost prohibitive. Computational approaches for informing experimentalists where to allocate wet-lab resources are available, including a variety of machine learning models. Assessing the accuracy of machine learning models for predicting the effects of mutations is dependent on experiments for amino acid substitutions performed in vitro. When similar experiments on physical proteins have been performed by multiple laboratories, the use of the data near the juncture of stabilizing and destabilizing mutations is questionable. In this work, we explore a systematic and principled alternative to discarding experimental data close to the juncture of stabilizing and destabilizing mutations. We model the inconclusive range of experimental [Formula: see text] values via 3- and 5-way classifiers, and systematically explore potential boundaries for the range of inconclusive experimental values. We demonstrate the effectiveness of potential boundaries through confusion matrices and heat map visualizations. We explore two novel metrics for assessing viable cutoff ranges, and find that under these metrics, a lower cutoff near [Formula: see text] and an upper cutoff near [Formula: see text] are optimal across multiple machine learning models.

scholarly journals Machine learning methods for stream water temperature prediction

2021 ◽  
Author(s):  
Moritz Feigl ◽  
Katharina Lebiedzinski ◽  
Mathew Herrnegger ◽  
Karsten Schulz
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Linear Regression ◽  
Water Temperature ◽  
Stream Water ◽  
Easy Access ◽  
Learning Models ◽  
Temperature Prediction ◽  
Wide Range ◽  
Machine Learning Models

Abstract. Water temperature in rivers is a crucial environmental factor with the ability to alter hydro-ecological as well as socio-economic conditions within a catchment. The development of modelling concepts for predicting river water temperature is and will be essential for an effective integrated water management and the development of adaptation strategies to future global changes (e.g. climate change). This study tests the performance of 6 different machine learning models: step-wise linear regression, Random forest, eXtreme Gradient Boosting (XGBoost), Feedforward neural networks (FNN), and two types of Recurrent neural networks (RNN). All models are applied using different data inputs for daily water temperature prediction in 10 Austrian catchments ranging from 200 km2 to 96000 km2 and exhibiting a wide range of physiographic characteristics. The evaluated input data sets include combinations of daily means of air temperature, runoff, precipitation and global radiation. Bayesian optimization is applied to optimize the hyperparameters of all applied machine learning models. To make the results comparable to previous studies, two widely used benchmark models are applied additionally: linear regression and air2stream. With a mean root mean squared error (RMSE) of 0.55 °C the tested models could significantly improve water temperature prediction compared to linear regression (1.55 °C) and air2stream (0.98 °C). In general, the results show a very similar performance of the tested machine learning models, with a median RMSE difference of 0.08 °C between the models. From the 6 tested machine learning models both FNNs and XGBoost performed best in 4 of the 10 catchments. RNNs are the best performing models in the largest catchment, indicating that RNNs are mainly performing well when processes with long-term dependencies are important. Furthermore, a wide range of performance was observed for different hyperparameter sets for the tested models, showing the importance of hyperprameter optimization. Especially the FNN model results showed an extremely large RMSE standard deviation of 1.60 °C due to the chosen hyperparamerters. This study evaluates different sets of input variables, machine learning models and training characteristics for daily stream water temperature prediction, acting as a basis for future development of regional multi-catchment water temperature prediction models. All preprocessing steps and models are implemented into the open source R package wateRtemp, to provide easy access to these modelling approaches and facilitate further research.

Sign in / Sign up

Export Citation Format

Share Document