scholarly journals A comparison of methods for interpreting random forest models of genetic association in the presence of non-additive interactions

2021 ◽  
Vol 14 (1) ◽  
Author(s):  
Alena Orlenko ◽  
Jason H. Moore
Keyword(s):  
Random Forest ◽  
Genetic Association ◽  
Simulated Data ◽  
Machine Learning Algorithms ◽  
Theory Approach ◽  
Rank Estimation ◽  
Feature Importance ◽  
Random Forest Models ◽  
Additive Interactions ◽  
Real World Datasets

Abstract Background Non-additive interactions among genes are frequently associated with a number of phenotypes, including known complex diseases such as Alzheimer’s, diabetes, and cardiovascular disease. Detecting interactions requires careful selection of analytical methods, and some machine learning algorithms are unable or underpowered to detect or model feature interactions that exhibit non-additivity. The Random Forest method is often employed in these efforts due to its ability to detect and model non-additive interactions. In addition, Random Forest has the built-in ability to estimate feature importance scores, a characteristic that allows the model to be interpreted with the order and effect size of the feature association with the outcome. This characteristic is very important for epidemiological and clinical studies where results of predictive modeling could be used to define the future direction of the research efforts. An alternative way to interpret the model is with a permutation feature importance metric which employs a permutation approach to calculate a feature contribution coefficient in units of the decrease in the model’s performance and with the Shapely additive explanations which employ cooperative game theory approach. Currently, it is unclear which Random Forest feature importance metric provides a superior estimation of the true informative contribution of features in genetic association analysis. Results To address this issue, and to improve interpretability of Random Forest predictions, we compared different methods for feature importance estimation in real and simulated datasets with non-additive interactions. As a result, we detected a discrepancy between the metrics for the real-world datasets and further established that the permutation feature importance metric provides more precise feature importance rank estimation for the simulated datasets with non-additive interactions. Conclusions By analyzing both real and simulated data, we established that the permutation feature importance metric provides more precise feature importance rank estimation in the presence of non-additive interactions.

scholarly journals A comparison of methods for interpreting random forest models of genetic association in the presence of non-additive interactions.

2021 ◽  
Author(s):  
Alena Orlenko ◽  
Jason H Moore
Keyword(s):  
Random Forest ◽  
Genetic Association ◽  
Simulated Data ◽  
Machine Learning Algorithms ◽  
Theory Approach ◽  
Rank Estimation ◽  
Feature Importance ◽  
Random Forest Models ◽  
Additive Interactions ◽  
Real World Datasets

Abstract Background: Non-additive interactions among genes are frequently associated with a number of phenotypes, including known complex diseases such as Alzheimer’s, diabetes, and cardiovascular disease. Detecting interactions requires careful selection of analytical methods, and some machine learning algorithms are unable or underpowered to detect or model feature interactions that exhibit non-additivity. The Random Forest method is often employed in these efforts due to its ability to detect and model non-additive interactions. In addition, Random Forest has the built-in ability to estimate feature importance scores, a characteristic that allows the model to be interpreted with the order and effect size of the feature association with the outcome. This characteristic is very important for epidemiological and clinical studies where results of predictive modeling could be used to define the future direction of the research efforts. An alternative way to interpret the model is with a permutation feature importance metric which employs a permutation approach to calculate a feature contribution coefficient in units of the decrease in the model’s performance and with the Shapely additive explanations which employ cooperative game theory approach. Currently, it is unclear which Random Forest feature importance metric provides a superior estimation of the true informative contribution of features in genetic association analysis. Results: To address this issue, and to improve interpretability of Random Forest predictions, we compared different methods for feature importance estimation in real and simulated datasets with non-additive interactions. As a result, we detected a discrepancy between the metrics for the real-world datasets and further established that the permutation feature importance metric provides more precise feature importance rank estimation for the simulated datasets with non-additive interactions. Conclusions: By analyzing both real and simulated data, we established that the permutation feature importance metric provides more precise feature importance rank estimation in the presence of non-additive interactions.

scholarly journals A comparison of methods for interpreting random forest models of genetic association in the presence of non-additive interactions.

2020 ◽  
Author(s):  
Alena Orlenko ◽  
Jason H Moore
Keyword(s):  
Random Forest ◽  
Genetic Association ◽  
Simulated Data ◽  
Machine Learning Algorithms ◽  
Theory Approach ◽  
Rank Estimation ◽  
Feature Importance ◽  
Random Forest Models ◽  
Additive Interactions ◽  
Real World Datasets

Abstract Background: Non-additive interactions among genes are frequently associated with a number of phenotypes, including known complex diseases such as Alzheimer’s, diabetes, and cardiovascular disease. Detecting interactions requires careful selection of analytical methods, and some machine learning algorithms are unable or underpowered to detect or model feature interactions that exhibit non-additivity. The Random Forest method is often employed in these efforts due to its ability to detect and model non-additive interactions. In addition, Random Forest has the built-in ability to estimate feature importance scores, a characteristic that allows the model to be interpreted with the order and effect size of the feature association with the outcome. This characteristic is very important for epidemiological and clinical studies where results of predictive modeling could be used to define the future direction of the research efforts. An alternative way to interpret the model is with a permutation feature importance metric which employs a permutation approach to calculate a feature contribution coefficient in units of the decrease in the model’s performance and with the Shapely additive explanations which employ cooperative game theory approach. Currently, it is unclear which Random Forest feature importance metric provides a superior estimation of the true informative contribution of features in genetic association analysis. Results: To address this issue, and to improve interpretability of Random Forest predictions, we compared different methods for feature importance estimation in real and simulated datasets with non-additive interactions. As a result, we detected a discrepancy between the metrics for the real-world datasets and further established that the permutation feature importance metric provides more precise feature importance rank estimation for the simulated datasets with non-additive interactions. Conclusions: By analyzing both real and simulated data, we established that the permutation feature importance metric provides more precise feature importance rank estimation in the presence of non-additive interactions.

scholarly journals Improving the interpretability of random forest models of genetic association in the presence of non-additive interactions

2020 ◽  
Author(s):  
Alena Orlenko ◽  
Jason H Moore
Keyword(s):  
Random Forest ◽  
Genetic Association ◽  
Simulated Data ◽  
Machine Learning Algorithms ◽  
Rank Estimation ◽  
Feature Importance ◽  
Random Forest Models ◽  
Additive Interactions ◽  
Real World Datasets ◽  
Future Direction

Abstract Background: Non-additive interactions among genes are frequently associated with a number of phenotypes, including known complex diseases such as Alzheimer’s, diabetes, and cardiovascular disease. Detecting interactions requires careful selection of analytical methods, and some machine learning algorithms are unable or underpowered to detect or model feature interactions that exhibit non-additivity. The Random Forest method is often employed in these efforts due to its ability to detect and model non-additive interactions. In addition, Random Forest has the built-in ability to estimate feature importance scores, a characteristic that allows the model to be interpreted with the order and effect size of the feature association with the outcome. This characteristic is very important for epidemiological and clinical studies where results of predictive modeling could be used to define the future direction of the research efforts. An alternative way to interpret the model is with a permutation feature importance metric which employs a permutation approach to calculate a feature contribution coefficient in units of the decrease in the model’s performance. Currently, it is unclear which Random Forest feature importance metric provides a superior estimation of the true informative contribution of features in genetic association analysis. Results: To address this issue, and to improve interpretability of Random Forest predictions, we compared different methods for feature importance estimation in real and simulated datasets with non-additive interactions. As a result, we detected a discrepancy between the metrics evaluations for the real-world datasets and further established that the permutation feature importance metric provides more precise feature importance rank estimation for the simulated datasets with non-additive interactions. Conclusions: By analyzing both real and simulated data, we established that the permutation feature importance metric provides more precise feature importance rank estimation in the presence of non-additive interactions.

scholarly journals Improving the Interpretability of Random Forest Models of Genetic Association in the Presence of Non-Additive Interactions

2020 ◽  
Author(s):  
Alena Orlenko ◽  
Jason H Moore
Keyword(s):  
Random Forest ◽  
Genetic Association ◽  
Simulated Data ◽  
Machine Learning Algorithms ◽  
Rank Estimation ◽  
Feature Importance ◽  
Random Forest Models ◽  
Additive Interactions ◽  
Real World Datasets ◽  
Future Direction

Abstract Background Non-additive interactions among genes are frequently associated with a number of phenotypes, including known complex diseases such as Alzheimer’s, diabetes, and cardiovascular disease. Detecting interactions requires careful selection of analytical methods, and some machine learning algorithms are unable or underpowered to detect or model feature interactions that exhibit non-additivity. The Random Forest method is often employed in these efforts due to its ability to detect and model non-additive interactions. In addition, Random Forest has the built-in ability to estimate feature importance scores, a characteristic that allows the model to be interpreted with the order and effect size of the feature association with the outcome. This characteristic is very important for epidemiological and clinical studies where results of predictive modeling could be used to define the future direction of the research efforts. An alternative way to interpret the model is with a permutation feature importance metric which employs a permutation approach to calculate a feature contribution coefficient in units of the decrease in the model’s performance. Currently, it is unclear which Random Forest feature importance metric provides a superior estimation of the true informative contribution of features in genetic association analysis.Results To address this issue, and to improve interpretability of Random Forest predictions, we compared different methods for feature importance estimation in real and simulated datasets with non-additive interactions. As a result, we detected a discrepancy between the metrics evaluations for the real-world datasets and further established that the permutation feature importance metric provides more precise feature importance rank estimation for the simulated datasets with non-additive interactions.Conclusions By analyzing both real and simulated data, we established that the permutation feature importance metric provides more precise feature importance rank estimation in the presence of non-additive interactions.

scholarly journals Predicting the animal hosts of coronaviruses from compositional biases of spike protein and whole genome sequences through machine learning

2020 ◽  
Author(s):  
Liam Brierley ◽  
Anna Fowler
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Machine Learning Algorithms ◽  
Spike Protein ◽  
Animal Origin ◽  
Whole Genome ◽  
Genome Sequences ◽  
Genome Composition ◽  
Forest Models ◽  
Random Forest Models

AbstractThe COVID-19 pandemic has demonstrated the serious potential for novel zoonotic coronaviruses to emerge and cause major outbreaks. The immediate animal origin of the causative virus, SARS-CoV-2, remains unknown, a notoriously challenging task for emerging disease investigations. Coevolution with hosts leads to specific evolutionary signatures within viral genomes that can inform likely animal origins. We obtained a set of 650 spike protein and 511 whole genome nucleotide sequences from 225 and 187 viruses belonging to the family Coronaviridae, respectively. We then trained random forest models independently on genome composition biases of spike protein and whole genome sequences, including dinucleotide and codon usage biases in order to predict animal host (of nine possible categories, including human). In hold-one-out cross-validation, predictive accuracy on unseen coronaviruses consistently reached ∼73%, indicating evolutionary signal in spike proteins to be just as informative as whole genome sequences. However, different composition biases were informative in each case. Applying optimised random forest models to classify human sequences of MERS-CoV and SARS-CoV revealed evolutionary signatures consistent with their recognised intermediate hosts (camelids, carnivores), while human sequences of SARS-CoV-2 were predicted as having bat hosts (suborder Yinpterochiroptera), supporting bats as the suspected origins of the current pandemic. In addition to phylogeny, variation in genome composition can act as an informative approach to predict emerging virus traits as soon as sequences are available. More widely, this work demonstrates the potential in combining genetic resources with machine learning algorithms to address long-standing challenges in emerging infectious diseases.

scholarly journals Comparing Boosting and Bagging for Decision Trees of Rankings

2021 ◽  
Author(s):  
Antonella Plaia ◽  
Simona Buscemi ◽  
Johannes Fürnkranz ◽  
Eneldo Loza Mencía
Keyword(s):  
Decision Trees ◽  
Ordinal Data ◽  
Simulated Data ◽  
Ensemble Methods ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Real World Datasets ◽  
Definition Of ◽  
Multiple Trees ◽  
Prediction Problems

AbstractDecision tree learning is among the most popular and most traditional families of machine learning algorithms. While these techniques excel in being quite intuitive and interpretable, they also suffer from instability: small perturbations in the training data may result in big changes in the predictions. The so-called ensemble methods combine the output of multiple trees, which makes the decision more reliable and stable. They have been primarily applied to numeric prediction problems and to classification tasks. In the last years, some attempts to extend the ensemble methods to ordinal data can be found in the literature, but no concrete methodology has been provided for preference data. In this paper, we extend decision trees, and in the following also ensemble methods to ranking data. In particular, we propose a theoretical and computational definition of bagging and boosting, two of the best known ensemble methods. In an experimental study using simulated data and real-world datasets, our results confirm that known results from classification, such as that boosting outperforms bagging, could be successfully carried over to the ranking case.

scholarly journals Predicting the animal hosts of coronaviruses from compositional biases of spike protein and whole genome sequences through machine learning

2021 ◽  
Vol 17 (4) ◽  
pp. e1009149
Author(s):  
Liam Brierley ◽  
Anna Fowler
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Machine Learning Algorithms ◽  
Spike Protein ◽  
Animal Origin ◽  
Whole Genome ◽  
Genome Sequences ◽  
Genome Composition ◽  
Forest Models ◽  
Random Forest Models

The COVID-19 pandemic has demonstrated the serious potential for novel zoonotic coronaviruses to emerge and cause major outbreaks. The immediate animal origin of the causative virus, SARS-CoV-2, remains unknown, a notoriously challenging task for emerging disease investigations. Coevolution with hosts leads to specific evolutionary signatures within viral genomes that can inform likely animal origins. We obtained a set of 650 spike protein and 511 whole genome nucleotide sequences from 222 and 185 viruses belonging to the family Coronaviridae, respectively. We then trained random forest models independently on genome composition biases of spike protein and whole genome sequences, including dinucleotide and codon usage biases in order to predict animal host (of nine possible categories, including human). In hold-one-out cross-validation, predictive accuracy on unseen coronaviruses consistently reached ~73%, indicating evolutionary signal in spike proteins to be just as informative as whole genome sequences. However, different composition biases were informative in each case. Applying optimised random forest models to classify human sequences of MERS-CoV and SARS-CoV revealed evolutionary signatures consistent with their recognised intermediate hosts (camelids, carnivores), while human sequences of SARS-CoV-2 were predicted as having bat hosts (suborder Yinpterochiroptera), supporting bats as the suspected origins of the current pandemic. In addition to phylogeny, variation in genome composition can act as an informative approach to predict emerging virus traits as soon as sequences are available. More widely, this work demonstrates the potential in combining genetic resources with machine learning algorithms to address long-standing challenges in emerging infectious diseases.

scholarly journals The Four Cosmic Tidal Web Elements from the β-skeleton

2021 ◽  
Vol 922 (2) ◽  
pp. 204
Author(s):  
John F. Suárez-Pérez ◽  
Yeimy Camargo ◽  
Xiao-Dong Li ◽  
Jaime E. Forero-Romero
Keyword(s):  
Machine Learning ◽  
Dark Matter ◽  
Random Forest ◽  
Machine Learning Algorithms ◽  
Skeleton Graph ◽  
Random Forest Models ◽  
Web Classification ◽  
Global Accuracy ◽  
Future Work

Abstract Precise cosmic web classification of observed galaxies in massive spectroscopic surveys can be either highly uncertain or computationally expensive. As an alternative, we explore a fast Machine Learning-based approach to infer the underlying dark matter tidal cosmic web environment of a galaxy distribution from its β-skeleton graph. We develop and test our methodology using the cosmological magnetohydrodynamic simulation Illustris-TNG at z = 0. We explore three different tree-based machine-learning algorithms to find that a random forest classifier can best use graph-based features to classify a galaxy as belonging to a peak, filament, or sheet as defined by the T-Web classification algorithm. The best match between the galaxies and the dark matter T-Web corresponds to a density field smoothed over scales of 2 Mpc, a threshold over the eigenvalues of the dimensionless tidal tensor of λ th = 0.0, and galaxy number densities around 8 × 10−3 Mpc−3. This methodology results on a weighted F1 score of 0.728 and a global accuracy of 74%. More extensive tests that take into account light-cone effects and redshift space distortions are left for future work. We make one of our highest ranking random forest models available on a public repository for future reference and reuse.

Development of Prediction Models Using Machine Learning Algorithms for Girls with Suspected Central Precocious Puberty: Retrospective Study (Preprint)

2018 ◽  
Author(s):  
Liyan Pan ◽  
Guangjian Liu ◽  
Xiaojian Mao ◽  
Huixian Li ◽  
Jiexin Zhang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Retrospective Study ◽  
Random Forest ◽  
Precocious Puberty ◽  
Prediction Models ◽  
Central Precocious Puberty ◽  
Machine Learning Algorithms ◽  
Stimulation Test ◽  
Gnrh Analogue ◽  
Prediction Probability

BACKGROUND Central precocious puberty (CPP) in girls seriously affects their physical and mental development in childhood. The method of diagnosis—gonadotropin-releasing hormone (GnRH)–stimulation test or GnRH analogue (GnRHa)–stimulation test—is expensive and makes patients uncomfortable due to the need for repeated blood sampling. OBJECTIVE We aimed to combine multiple CPP–related features and construct machine learning models to predict response to the GnRHa-stimulation test. METHODS In this retrospective study, we analyzed clinical and laboratory data of 1757 girls who underwent a GnRHa test in order to develop XGBoost and random forest classifiers for prediction of response to the GnRHa test. The local interpretable model-agnostic explanations (LIME) algorithm was used with the black-box classifiers to increase their interpretability. We measured sensitivity, specificity, and area under receiver operating characteristic (AUC) of the models. RESULTS Both the XGBoost and random forest models achieved good performance in distinguishing between positive and negative responses, with the AUC ranging from 0.88 to 0.90, sensitivity ranging from 77.91% to 77.94%, and specificity ranging from 84.32% to 87.66%. Basal serum luteinizing hormone, follicle-stimulating hormone, and insulin-like growth factor-I levels were found to be the three most important factors. In the interpretable models of LIME, the abovementioned variables made high contributions to the prediction probability. CONCLUSIONS The prediction models we developed can help diagnose CPP and may be used as a prescreening tool before the GnRHa-stimulation test.

scholarly journals Exploratory Analysis of Driving Force of Wildfires in Australia: An Application of Machine Learning within Google Earth Engine

2020 ◽  
Vol 13 (1) ◽  
pp. 10
Author(s):  
Andrea Sulova ◽  
Jamal Jokar Arsanjani
Keyword(s):  
Climate Change ◽  
Machine Learning ◽  
Random Forest ◽  
Google Earth ◽  
Summer Season ◽  
Driving Factors ◽  
Machine Learning Algorithms ◽  
Classification And Regression Tree ◽  
Training Dataset ◽  
Google Earth Engine

Recent studies have suggested that due to climate change, the number of wildfires across the globe have been increasing and continue to grow even more. The recent massive wildfires, which hit Australia during the 2019–2020 summer season, raised questions to what extent the risk of wildfires can be linked to various climate, environmental, topographical, and social factors and how to predict fire occurrences to take preventive measures. Hence, the main objective of this study was to develop an automatized and cloud-based workflow for generating a training dataset of fire events at a continental level using freely available remote sensing data with a reasonable computational expense for injecting into machine learning models. As a result, a data-driven model was set up in Google Earth Engine platform, which is publicly accessible and open for further adjustments. The training dataset was applied to different machine learning algorithms, i.e., Random Forest, Naïve Bayes, and Classification and Regression Tree. The findings show that Random Forest outperformed other algorithms and hence it was used further to explore the driving factors using variable importance analysis. The study indicates the probability of fire occurrences across Australia as well as identifies the potential driving factors of Australian wildfires for the 2019–2020 summer season. The methodical approach and achieved results and drawn conclusions can be of great importance to policymakers, environmentalists, and climate change researchers, among others.

Sign in / Sign up

Export Citation Format

Share Document