A data-driven approach based on long short-term memory and hidden Markov model for crack propagation prediction

Encounter risk prediction is critical for safe ship navigation, especially in congested waters, where ships sail very near to each other during various encounter situations. Prior studies on the risk of ship collisions were unable to address the uncertainty of the encounter process when ignoring the complex motions constituting the dynamic ship encounter behavior, which may seriously affect the risk prediction performance. To fill this gap, a novel AIS data-driven approach is proposed for ship encounter risk prediction by modeling intership behavior patterns. In particular, multidimensional features of intership behaviors are extracted from the AIS trace data to capture spatial dependencies between encountering ships. Then, the challenging task of risk prediction is to discover the complex and uncertain relationship between intership behaviors and future collision risk. To address this issue, we propose a deep learning framework. To represent the temporal dynamics of the encounter process, we use the sliding window technique to generate the sequences of behavioral features. The collision risk level at a future time is taken as the class label of the sequence. Then, the long short-term memory network, which has a strong ability to model temporal dependency and complex patterns, is extended to establish the relationship. The benefit of our approach is that it transforms the complex problem for risk prediction into a time series classification task, which makes collision risk prediction reliable and easier to implement. Experiments were conducted on a set of naturalistic data from various encounter scenarios in the South Channel of the Yangtze River Estuary. The results show that the proposed data-driven approach can predict future collision risk with high accuracy and efficiency. The approach is expected to be applied for the early prediction of encountering ships and as decision support to improve navigation safety.

Download Full-text

Forecasting Covid-19 Dynamics in Brazil: A Data Driven Approach

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph17145115 ◽

2020 ◽

Vol 17 (14) ◽

pp. 5115 ◽

Cited By ~ 2

Author(s):

Igor Gadelha Pereira ◽

Joris Michel Guerin ◽

Andouglas Gonçalves Silva Júnior ◽

Gabriel Santos Garcia ◽

Prisco Piscitelli ◽

...

Keyword(s):

Short Term Memory ◽

Data Driven ◽

Advanced Stage ◽

Short Term ◽

Term Memory ◽

Future Data ◽

Data Driven Approach ◽

Ann Models ◽

Preliminary Study ◽

Long Short Term Memory

The contribution of this paper is twofold. First, a new data driven approach for predicting the Covid-19 pandemic dynamics is introduced. The second contribution consists in reporting and discussing the results that were obtained with this approach for the Brazilian states, with predictions starting as of 4 May 2020. As a preliminary study, we first used an Long Short Term Memory for Data Training-SAE (LSTM-SAE) network model. Although this first approach led to somewhat disappointing results, it served as a good baseline for testing other ANN types. Subsequently, in order to identify relevant countries and regions to be used for training ANN models, we conduct a clustering of the world’s regions where the pandemic is at an advanced stage. This clustering is based on manually engineered features representing a country’s response to the early spread of the pandemic, and the different clusters obtained are used to select the relevant countries for training the models. The final models retained are Modified Auto-Encoder networks, that are trained on these clusters and learn to predict future data for Brazilian states. These predictions are used to estimate important statistics about the disease, such as peaks and number of confirmed cases. Finally, curve fitting is carried out to find the distribution that best fits the outputs of the MAE, and to refine the estimates of the peaks of the pandemic. Predicted numbers reach a total of more than one million infected Brazilians, distributed among the different states, with São Paulo leading with about 150 thousand confirmed cases predicted. The results indicate that the pandemic is still growing in Brazil, with most states peaks of infection estimated in the second half of May 2020. The estimated end of the pandemics (97% of cases reaching an outcome) spread between June and the end of August 2020, depending on the states.

Download Full-text

Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

Nonlinear Processes in Geophysics ◽

10.5194/npg-27-373-2020 ◽

2020 ◽

Vol 27 (3) ◽

pp. 373-389 ◽

Cited By ~ 7

Author(s):

Ashesh Chattopadhyay ◽

Pedram Hassanzadeh ◽

Devika Subramanian

Keyword(s):

Neural Network ◽

Machine Learning ◽

Short Term Memory ◽

Data Driven ◽

Reservoir Computing ◽

Short Term ◽

Term Memory ◽

Machine Learning Methods ◽

Long Short Term Memory ◽

Lorenz 96

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

Download Full-text

Data-Driven Phone Selection for Language Identification via Bidirectional Long Short-Term Memory Modeling

Communications in Computer and Information Science - Computational Intelligence and Intelligent Systems ◽

10.1007/978-981-13-1648-7_26 ◽

2018 ◽

pp. 301-312

Author(s):

Xiao Song ◽

Qiang Cheng ◽

Jingping Xing ◽

Yuexian Zou

Keyword(s):

Short Term Memory ◽

Language Identification ◽

Data Driven ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Selection For ◽

Memory Modeling

Download Full-text

Efficient and data-driven prediction of water breakthrough in subsurface systems using deep long short-term memory machine learning

Computational Geosciences ◽

10.1007/s10596-020-10005-2 ◽

2020 ◽

Author(s):

Tao Bai ◽

Pejman Tahmasebi

Keyword(s):

Machine Learning ◽

Short Term Memory ◽

Data Driven ◽

Short Term ◽

Term Memory ◽

Water Breakthrough ◽

Long Short Term Memory

Download Full-text

Data-Driven Predictive Modeling of Neuronal Dynamics Using Long Short-Term Memory

Algorithms ◽

10.3390/a12100203 ◽

2019 ◽

Vol 12 (10) ◽

pp. 203

Author(s):

Benjamin Plaster ◽

Gautam Kumar

Keyword(s):

Neural Network ◽

Time Horizon ◽

Short Term Memory ◽

Temporal Dynamics ◽

Data Driven ◽

Brain Dynamics ◽

Short Term ◽

Term Memory ◽

Brain Functions ◽

Long Short Term Memory

Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions have been of interest to engineers, mathematicians and physicists over the last several decades. With the motivation of developing computationally efficient models of brain dynamics to use in designing control-theoretic neurostimulation strategies, we have developed a novel data-driven approach in a long short-term memory (LSTM) neural network architecture to predict the temporal dynamics of complex systems over an extended long time-horizon in future. In contrast to recent LSTM-based dynamical modeling approaches that make use of multi-layer perceptrons or linear combination layers as output layers, our architecture uses a single fully connected output layer and reversed-order sequence-to-sequence mapping to improve short time-horizon prediction accuracy and to make multi-timestep predictions of dynamical behaviors. We demonstrate the efficacy of our approach in reconstructing the regular spiking to bursting dynamics exhibited by an experimentally-validated 9-dimensional Hodgkin-Huxley model of hippocampal CA1 pyramidal neurons. Through simulations, we show that our LSTM neural network can predict the multi-time scale temporal dynamics underlying various spiking patterns with reasonable accuracy. Moreover, our results show that the predictions improve with increasing predictive time-horizon in the multi-timestep deep LSTM neural network.

Download Full-text

The performance of LSTM models from basin to continental scales

10.5194/egusphere-egu2020-8855 ◽

2020 ◽

Author(s):

Frederik Kratzert ◽

Daniel Klotz ◽

Günter Klambauer ◽

Grey Nearing ◽

Sepp Hochreiter

Keyword(s):

Machine Learning ◽

Large Scale ◽

Short Term Memory ◽

Regional Scale ◽

Data Driven ◽

Hydrological Models ◽

Short Term ◽

Term Memory ◽

Basin Scale ◽

Long Short Term Memory

Simulation accuracy among traditional hydrological models usually degrades significantly when going from single basin to regional scale. Hydrological models perform best when calibrated for specific basins, and do worse when a regional calibration scheme is used.&#160;One reason for this is that these models do not (have to) learn hydrological processes from data. Rather, they have a predefined model structure and only a handful of parameters adapt to specific basins. This often yields less-than-optimal parameter values when the loss is not determined by a single basin, but by many through regional calibration.The opposite is true for data driven approaches where models tend to get better with more and diverse training data. We examine whether this holds true when modeling rainfall-runoff processes with deep learning, or if, like their process-based counterparts, data-driven hydrological models degrade when going from basin to regional scale.Recently, Kratzert et al. (2018) showed that the Long Short-Term Memory network (LSTM), a special type of recurrent neural network, achieves comparable performance to the SAC-SMA at basin scale. In follow up work Kratzert et al. (2019a) trained a single LSTM for hundreds of basins in the continental US, which outperformed a set of hydrological models significantly, even compared to basin-calibrated hydrological models. On average, a single LSTM is even better in out-of-sample predictions (ungauged) compared to the SAC-SMA in-sample (gauged) or US National Water Model (Kratzert et al. 2019b).LSTM-based approaches usually involve tuning a large number of hyperparameters, such as the number of neurons, number of layers, and learning rate, that are critical for the predictive performance. Therefore, large-scale hyperparameter search has to be performed to obtain a proficient LSTM network.&#160;&#160;However, in the abovementioned studies, hyperparameter optimization was not conducted at large scale and e.g. in Kratzert et al. (2018) the same network hyperparameters were used in all basins, instead of tuning hyperparameters for each basin separately. It is yet unclear whether LSTMs follow the same trend of traditional hydrological models to degrade performance from basin to regional scale.&#160;In the current study, we performed a computational expensive, basin-specific hyperparameter search to explore how site-specific LSTMs differ in performance compared to regionally calibrated LSTMs. We compared our results to the mHM and VIC models, once calibrated per-basin and once using an MPR regionalization scheme. These benchmark models were calibrated individual research groups, to eliminate bias in our study. We analyse whether differences in basin-specific vs regional model performance can be linked to basin attributes or data set characteristics.References:Kratzert, F., Klotz, D., Brenner, C., Schulz, K., and Herrnegger, M.: Rainfall&#8211;runoff modelling using Long Short-Term Memory (LSTM) networks, Hydrol. Earth Syst. Sci., 22, 6005&#8211;6022, https://doi.org/10.5194/hess-22-6005-2018, 2018.&#160;Kratzert, F., Klotz, D., Shalev, G., Klambauer, G., Hochreiter, S., and Nearing, G.: Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets, Hydrol. Earth Syst. Sci., 23, 5089&#8211;5110, https://doi.org/10.5194/hess-23-5089-2019, 2019a.&#160;Kratzert, F., Klotz, D., Herrnegger, M., Sampson, A. K., Hochreiter, S., & Nearing, G. S.: Toward improved predictions in ungauged basins: Exploiting the power of machine learning. Water Resources Research, 55. https://doi.org/10.1029/2019WR026065, 2019b.

Download Full-text

Data-Driven Predictive Modeling of Neuronal Dynamics using Long Short-Term Memory

10.20944/preprints201908.0155.v2 ◽

2019 ◽

Author(s):

Benjamin Plaster ◽

Gautam Kumar

Keyword(s):

Neural Network ◽

Time Horizon ◽

Short Term Memory ◽

Temporal Dynamics ◽

Data Driven ◽

Brain Dynamics ◽

Short Term ◽

Term Memory ◽

Brain Functions ◽

Long Short Term Memory

Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of developing computationally efficient models of brain dynamics to use in designing control-theoretic neurostimulation strategies, we have developed a novel data-driven approach in a long short-term memory (LSTM) neural network architecture to predict the temporal dynamics of complex systems over an extended long time-horizon in future. In contrast to recent LSTM-based dynamical modeling approaches that make use of multi-layer perceptrons or linear combination layers as output layers, our architecture uses a single fully connected output layer and reversed-order sequence-to-sequence mapping to improve short time-horizon prediction accuracy and to make multi-timestep predictions of dynamical behaviors. We demonstrate the efficacy of our approach in reconstructing the regular spiking to bursting dynamics exhibited by an experimentally-validated 9-dimensional Hodgkin-Huxley model of hippocampal CA1 pyramidal neurons. Through simulations, we show that our LSTM neural network can predict the multi-time scale temporal dynamics underlying various spiking patterns with reasonable accuracy. Moreover, our results show that the predictions improve with increasing predictive time-horizon in the multi-timestep deep LSTM neural network.

Download Full-text