Analysis of a bistable climate toy model with physics-based machine learning methods

AbstractWe propose a comprehensive framework able to address both the predictability of the first and of the second kind for high-dimensional chaotic models. For this purpose, we analyse the properties of a newly introduced multistable climate toy model constructed by coupling the Lorenz ’96 model with a zero-dimensional energy balance model. First, the attractors of the system are identified with Monte Carlo Basin Bifurcation Analysis. Additionally, we are able to detect the Melancholia state separating the two attractors. Then, Neural Ordinary Differential Equations are applied to predict the future state of the system in both of the identified attractors.

Download Full-text

Analysing Conceptual Climate Models with Monte Carlo Basin Bifurcation Analysis

10.5194/egusphere-egu2020-18301 ◽

2020 ◽

Author(s):

Maximilian Gelbrecht ◽

Jürgen Kurths ◽

Frank Hellmann

Keyword(s):

Monte Carlo ◽

Complex Systems ◽

Bifurcation Analysis ◽

Climate Models ◽

Climate Model ◽

Initial Conditions ◽

Balance Model ◽

High Dimensional ◽

Asymptotic State ◽

Wide Range

Many high-dimensional complex systems such as climate models exhibit an enormously complex landscape of possible asymptotic state. On most occasions these are challenging to analyse with traditional bifurcation analysis methods. Often, one is also more broadly interested in classes of asymptotic states. Here, we present a novel numerical approach prepared for analysing such high-dimensional multistable complex systems: Monte Carlo Basin Bifurcation Analysis (MCBB).&#160; Based on random sampling and clustering methods, we identify the type of dynamic regimes with the largest basins of attraction and track how the volume of these basins change with the system parameters. In order to due this suitable, easy to compute, statistics of trajectories with randomly generated initial conditions and parameters are clustered by an algorithm such as DBSCAN. Due to the modular and flexible nature of the method, it has a wide range of possible applications. While initially oscillator networks were one of the main applications of this methods, here we present an analysis of a simple conceptual climate model setup up by coupling an energy balance model to the Lorenz96 system. The method is available to use as a package for the Julia language.&#160;

Download Full-text

Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Nonlinear Processes in Geophysics ◽

10.5194/npg-27-373-2020 ◽

2020 ◽

Vol 27 (3) ◽

pp. 373-389 ◽

Cited By ~ 7

Author(s):

Ashesh Chattopadhyay ◽

Pedram Hassanzadeh ◽

Devika Subramanian

Keyword(s):

Neural Network ◽

Machine Learning ◽

Short Term Memory ◽

Data Driven ◽

Reservoir Computing ◽

Short Term ◽

Term Memory ◽

Machine Learning Methods ◽

Long Short Term Memory ◽

Lorenz 96

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

Download Full-text

A Comparison of Machine Learning Methods in a High-Dimensional Classification Problem

Business Systems Research Journal ◽

10.2478/bsrj-2014-0021 ◽

2014 ◽

Vol 5 (3) ◽

pp. 82-96 ◽

Cited By ~ 3

Author(s):

Marijana Zekić-Sušac ◽

Sanja Pfeifer ◽

Nataša Šarlija

Keyword(s):

Neural Network ◽

Machine Learning ◽

Classification Accuracy ◽

Classification Problem ◽

High Dimensional ◽

Nearest Neighbour ◽

Learning Methods ◽

Machine Learning Methods ◽

Dimensional Classification ◽

Artificial Neural

Abstract Background: Large-dimensional data modelling often relies on variable reduction methods in the pre-processing and in the post-processing stage. However, such a reduction usually provides less information and yields a lower accuracy of the model. Objectives: The aim of this paper is to assess the high-dimensional classification problem of recognizing entrepreneurial intentions of students by machine learning methods. Methods/Approach: Four methods were tested: artificial neural networks, CART classification trees, support vector machines, and k-nearest neighbour on the same dataset in order to compare their efficiency in the sense of classification accuracy. The performance of each method was compared on ten subsamples in a 10-fold cross-validation procedure in order to assess computing sensitivity and specificity of each model. Results: The artificial neural network model based on multilayer perceptron yielded a higher classification rate than the models produced by other methods. The pairwise t-test showed a statistical significance between the artificial neural network and the k-nearest neighbour model, while the difference among other methods was not statistically significant. Conclusions: Tested machine learning methods are able to learn fast and achieve high classification accuracy. However, further advancement can be assured by testing a few additional methodological refinements in machine learning methods.

Download Full-text

Variational Monte Carlo—bridging concepts of machine learning and high-dimensional partial differential equations

Advances in Computational Mathematics ◽

10.1007/s10444-019-09723-8 ◽

2019 ◽

Vol 45 (5-6) ◽

pp. 2503-2532 ◽

Cited By ~ 1

Author(s):

Martin Eigel ◽

Reinhold Schneider ◽

Philipp Trunschke ◽

Sebastian Wolf

Keyword(s):

Machine Learning ◽

Monte Carlo ◽

Partial Differential Equations ◽

Differential Equations ◽

High Dimensional ◽

Partial Differential ◽

Variational Monte Carlo

Download Full-text

TSLRF: Two-Stage Algorithm Based on Least Angle Regression and Random Forest in genome-wide association studies

Scientific Reports ◽

10.1038/s41598-019-54519-x ◽

2019 ◽

Vol 9 (1) ◽

Cited By ~ 1

Author(s):

Jiali Sun ◽

Qingtai Wu ◽

Dafeng Shen ◽

Yangjun Wen ◽

Fengrong Liu ◽

...

Keyword(s):

Machine Learning ◽

Random Forest ◽

The Other ◽

New Method ◽

Genome Wide Association ◽

High Dimensional ◽

Simulation Experiments ◽

Machine Learning Methods ◽

Least Angle Regression ◽

Genome Wide

AbstractOne of the most important tasks in genome-wide association analysis (GWAS) is the detection of single-nucleotide polymorphisms (SNPs) which are related to target traits. With the development of sequencing technology, traditional statistical methods are difficult to analyze the corresponding high-dimensional massive data or SNPs. Recently, machine learning methods have become more popular in high-dimensional genetic data analysis for their fast computation speed. However, most of machine learning methods have several drawbacks, such as poor generalization ability, over-fitting, unsatisfactory classification and low detection accuracy. This study proposed a two-stage algorithm based on least angle regression and random forest (TSLRF), which firstly considered the control of population structure and polygenic effects, then selected the SNPs that were potentially related to target traits by using least angle regression (LARS), furtherly analyzed this variable subset using random forest (RF) to detect quantitative trait nucleotides (QTNs) associated with target traits. The new method has more powerful detection in simulation experiments and real data analyses. The results of simulation experiments showed that, compared with the existing approaches, the new method effectively improved the detection ability of QTNs and model fitting degree, and required less calculation time. In addition, the new method significantly distinguished QTNs and other SNPs. Subsequently, the new method was applied to analyze five flowering-related traits in Arabidopsis. The results showed that, the distinction between QTNs and unrelated SNPs was more significant than the other methods. The new method detected 60 genes confirmed to be related to the target trait, which was significantly higher than the other methods, and simultaneously detected multiple gene clusters associated with the target trait.

Download Full-text

TADA: phylogenetic augmentation of microbiome samples enhances phenotype classification

Bioinformatics ◽

10.1093/bioinformatics/btz394 ◽

2019 ◽

Vol 35 (14) ◽

pp. i31-i40 ◽

Cited By ~ 1

Author(s):

Erfan Sayyari ◽

Ban Kawas ◽

Siavash Mirarab

Keyword(s):

Machine Learning ◽

Sample Size ◽

Data Augmentation ◽

Training Data ◽

Supplementary Information ◽

High Dimensional ◽

Learning Methods ◽

Machine Learning Methods ◽

Phenotype Classification ◽

Microbiome Data

Abstract Motivation Learning associations of traits with the microbial composition of a set of samples is a fundamental goal in microbiome studies. Recently, machine learning methods have been explored for this goal, with some promise. However, in comparison to other fields, microbiome data are high-dimensional and not abundant; leading to a high-dimensional low-sample-size under-determined system. Moreover, microbiome data are often unbalanced and biased. Given such training data, machine learning methods often fail to perform a classification task with sufficient accuracy. Lack of signal is especially problematic when classes are represented in an unbalanced way in the training data; with some classes under-represented. The presence of inter-correlations among subsets of observations further compounds these issues. As a result, machine learning methods have had only limited success in predicting many traits from microbiome. Data augmentation consists of building synthetic samples and adding them to the training data and is a technique that has proved helpful for many machine learning tasks. Results In this paper, we propose a new data augmentation technique for classifying phenotypes based on the microbiome. Our algorithm, called TADA, uses available data and a statistical generative model to create new samples augmenting existing ones, addressing issues of low-sample-size. In generating new samples, TADA takes into account phylogenetic relationships between microbial species. On two real datasets, we show that adding these synthetic samples to the training set improves the accuracy of downstream classification, especially when the training data have an unbalanced representation of classes. Availability and implementation TADA is available at https://github.com/tada-alg/TADA. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text

Sparse Boosting Based Machine Learning Methods for High-Dimensional Data

10.5772/intechopen.100506 ◽

2021 ◽

Author(s):

Mu Yue

Keyword(s):

Machine Learning ◽

Parameter Estimation ◽

Variable Selection ◽

Survival Data ◽

High Dimensional Data ◽

High Dimensional ◽

Learning Methods ◽

Require Time ◽

Machine Learning Methods ◽

Boosting Method

In high-dimensional data, penalized regression is often used for variable selection and parameter estimation. However, these methods typically require time-consuming cross-validation methods to select tuning parameters and retain more false positives under high dimensionality. This chapter discusses sparse boosting based machine learning methods in the following high-dimensional problems. First, a sparse boosting method to select important biomarkers is studied for the right censored survival data with high-dimensional biomarkers. Then, a two-step sparse boosting method to carry out the variable selection and the model-based prediction is studied for the high-dimensional longitudinal observations measured repeatedly over time. Finally, a multi-step sparse boosting method to identify patient subgroups that exhibit different treatment effects is studied for the high-dimensional dense longitudinal observations. This chapter intends to solve the problem of how to improve the accuracy and calculation speed of variable selection and parameter estimation in high-dimensional data. It aims to expand the application scope of sparse boosting and develop new methods of high-dimensional survival analysis, longitudinal data analysis, and subgroup analysis, which has great application prospects.

Download Full-text