Groundwater level modeling framework by combining the wavelet transform with a long short-term memory data-driven model

2021 ◽  
Vol 783 ◽  
pp. 146948
Author(s):  
Chengcheng Wu ◽  
Xiaoqin Zhang ◽  
Wanjie Wang ◽  
Chengpeng Lu ◽  
Yong Zhang ◽  
...  
Keyword(s):  
Wavelet Transform ◽  
Groundwater Level ◽  
Short Term Memory ◽  
Data Driven ◽  
Short Term ◽  
Modeling Framework ◽  
Term Memory ◽  
Long Short Term Memory

scholarly journals Improving the Accuracy of Dam Inflow Predictions Using a Long Short-Term Memory Network Coupled with Wavelet Transform and Predictor Selection

Mathematics ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. 551
Author(s):  
Trung Duc Tran ◽  
Vinh Ngoc Tran ◽  
Jongho Kim
Keyword(s):  
Wavelet Transform ◽  
Prediction Models ◽  
Short Term Memory ◽  
Random Search ◽  
Data Driven ◽  
Time Lags ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Dam Inflow

Accurate and reliable dam inflow prediction models are essential for effective reservoir operation and management. This study presents a data-driven model that couples a long short-term memory (LSTM) network with robust input predictor selection, input reconstruction by wavelet transformation, and efficient hyper-parameter optimization by K-fold cross-validation and the random search. First, a robust analysis using a “correlation threshold” for partial autocorrelation and cross-correlation functions is proposed, and only variables greater than this threshold are selected as input predictors and their time lags. This analysis indicates that a model trained on a threshold of 0.4 returns the highest Nash–Sutcliffe efficiency value; as a result, six principal inputs are selected. Second, using additional subseries reconstructed by the wavelet transform improves predictability, particularly for flow peak. The peak error values of LSTM with the transform are approximately one-half to one-quarter the size of those without the transform. Third, for a K of 5 as determined by the Silhouette coefficients and the distortion score, the wavelet-transformed LSTMs require a larger number of hidden units, epochs, dropout, and batch size. This complex configuration is needed because the amount of inputs used by these LSTMs is five times greater than that of other models. Last, an evaluation of accuracy performance reveals that the model proposed in this study, called SWLSTM, provides superior predictions of the daily inflow of the Hwacheon dam in South Korea compared with three other LSTM models by 84%, 78%, and 65%. These results strengthen the potential of data-driven models for efficient and effective reservoir inflow predictions, and should help policy-makers and operators better manage their reservoir operations.

scholarly journals Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network

2020 ◽  
Vol 27 (3) ◽  
pp. 373-389 ◽  
Author(s):  
Ashesh Chattopadhyay ◽  
Pedram Hassanzadeh ◽  
Devika Subramanian
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Short Term Memory ◽  
Data Driven ◽  
Reservoir Computing ◽  
Short Term ◽  
Term Memory ◽  
Machine Learning Methods ◽  
Long Short Term Memory ◽  
Lorenz 96

Abstract. In this paper, the performance of three machine-learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multiscale spatiotemporal Lorenz 96 system is examined. The methods are an echo state network (ESN, which is a type of reservoir computing; hereafter RC–ESN), a deep feed-forward artificial neural network (ANN), and a recurrent neural network (RNN) with long short-term memory (LSTM; hereafter RNN–LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale (X), intermediate (Y), and fast/small-scale (Z) processes. For training or testing, only X is available; Y and Z are never known or used. We show that RC–ESN substantially outperforms ANN and RNN–LSTM for short-term predictions, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps equivalent to several Lyapunov timescales. The RNN–LSTM outperforms ANN, and both methods show some prediction skills too. Furthermore, even after losing the trajectory, data predicted by RC–ESN and RNN–LSTM have probability density functions (pdf's) that closely match the true pdf – even at the tails. The pdf of the data predicted using ANN, however, deviates from the true pdf. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems, such as weather and climate, are discussed.

scholarly journals Analysis of Groundwater Level Variations Caused by the Changes in Groundwater Withdrawals Using Long Short-Term Memory Network

Hydrology ◽  
2020 ◽  
Vol 7 (3) ◽  
pp. 64 ◽  
Author(s):  
Mun-Ju Shin ◽  
Soo-Hyoung Moon ◽  
Kyung Goo Kang ◽  
Duk-Chul Moon ◽  
Hyuk-Joon Koh
Keyword(s):  
Groundwater Level ◽  
Short Term Memory ◽  
Groundwater Resources ◽  
Prediction Performance ◽  
Groundwater Withdrawal ◽  
Short Term ◽  
Maximum Increase ◽  
Term Memory ◽  
Long Short Term Memory ◽  
The Impact

To properly manage the groundwater resources, it is necessary to analyze the impact of groundwater withdrawal on the groundwater level. In this study, a Long Short-Term Memory (LSTM) network was used to evaluate the groundwater level prediction performance and analyze the impact of the change in the amount of groundwater withdrawal from the pumping wells on the change in the groundwater level in the nearby monitoring wells located in Jeju Island, Korea. The Nash–Sutcliffe efficiency between the observed and simulated groundwater level was over 0.97. Therefore, the groundwater prediction performance of LSTM was remarkably high. If the groundwater level is simulated on the assumption that the future withdrawal amount is reduced by 1/3 of the current groundwater withdrawal, the range of the maximum rise of the groundwater level would be 0.06–0.13 m compared to the current condition. In addition, assuming that no groundwater is taken, the range of the maximum increase in the groundwater level would be 0.11–0.38 m more than the current condition. Therefore, the effect of groundwater withdrawal on the groundwater level in this area was exceedingly small. The method and results can be used to develop new groundwater withdrawal sources for the redistribution of groundwater withdrawals.

scholarly journals Data-Driven Predictive Modeling of Neuronal Dynamics Using Long Short-Term Memory

Algorithms ◽  
2019 ◽  
Vol 12 (10) ◽  
pp. 203
Author(s):  
Benjamin Plaster ◽  
Gautam Kumar
Keyword(s):  
Neural Network ◽  
Time Horizon ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Data Driven ◽  
Brain Dynamics ◽  
Short Term ◽  
Term Memory ◽  
Brain Functions ◽  
Long Short Term Memory

Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions have been of interest to engineers, mathematicians and physicists over the last several decades. With the motivation of developing computationally efficient models of brain dynamics to use in designing control-theoretic neurostimulation strategies, we have developed a novel data-driven approach in a long short-term memory (LSTM) neural network architecture to predict the temporal dynamics of complex systems over an extended long time-horizon in future. In contrast to recent LSTM-based dynamical modeling approaches that make use of multi-layer perceptrons or linear combination layers as output layers, our architecture uses a single fully connected output layer and reversed-order sequence-to-sequence mapping to improve short time-horizon prediction accuracy and to make multi-timestep predictions of dynamical behaviors. We demonstrate the efficacy of our approach in reconstructing the regular spiking to bursting dynamics exhibited by an experimentally-validated 9-dimensional Hodgkin-Huxley model of hippocampal CA1 pyramidal neurons. Through simulations, we show that our LSTM neural network can predict the multi-time scale temporal dynamics underlying various spiking patterns with reasonable accuracy. Moreover, our results show that the predictions improve with increasing predictive time-horizon in the multi-timestep deep LSTM neural network.

The performance of LSTM models from basin to continental scales

2020 ◽  
Author(s):  
Frederik Kratzert ◽  
Daniel Klotz ◽  
Günter Klambauer ◽  
Grey Nearing ◽  
Sepp Hochreiter
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Short Term Memory ◽  
Regional Scale ◽  
Data Driven ◽  
Hydrological Models ◽  
Short Term ◽  
Term Memory ◽  
Basin Scale ◽  
Long Short Term Memory

<p>Simulation accuracy among traditional hydrological models usually degrades significantly when going from single basin to regional scale. Hydrological models perform best when calibrated for specific basins, and do worse when a regional calibration scheme is used. </p><p>One reason for this is that these models do not (have to) learn hydrological processes from data. Rather, they have a predefined model structure and only a handful of parameters adapt to specific basins. This often yields less-than-optimal parameter values when the loss is not determined by a single basin, but by many through regional calibration.</p><p>The opposite is true for data driven approaches where models tend to get better with more and diverse training data. We examine whether this holds true when modeling rainfall-runoff processes with deep learning, or if, like their process-based counterparts, data-driven hydrological models degrade when going from basin to regional scale.</p><p>Recently, Kratzert et al. (2018) showed that the Long Short-Term Memory network (LSTM), a special type of recurrent neural network, achieves comparable performance to the SAC-SMA at basin scale. In follow up work Kratzert et al. (2019a) trained a single LSTM for hundreds of basins in the continental US, which outperformed a set of hydrological models significantly, even compared to basin-calibrated hydrological models. On average, a single LSTM is even better in out-of-sample predictions (ungauged) compared to the SAC-SMA in-sample (gauged) or US National Water Model (Kratzert et al. 2019b).</p><p>LSTM-based approaches usually involve tuning a large number of hyperparameters, such as the number of neurons, number of layers, and learning rate, that are critical for the predictive performance. Therefore, large-scale hyperparameter search has to be performed to obtain a proficient LSTM network.  </p><p>However, in the abovementioned studies, hyperparameter optimization was not conducted at large scale and e.g. in Kratzert et al. (2018) the same network hyperparameters were used in all basins, instead of tuning hyperparameters for each basin separately. It is yet unclear whether LSTMs follow the same trend of traditional hydrological models to degrade performance from basin to regional scale. </p><p>In the current study, we performed a computational expensive, basin-specific hyperparameter search to explore how site-specific LSTMs differ in performance compared to regionally calibrated LSTMs. We compared our results to the mHM and VIC models, once calibrated per-basin and once using an MPR regionalization scheme. These benchmark models were calibrated individual research groups, to eliminate bias in our study. We analyse whether differences in basin-specific vs regional model performance can be linked to basin attributes or data set characteristics.</p><p>References:</p><p>Kratzert, F., Klotz, D., Brenner, C., Schulz, K., and Herrnegger, M.: Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks, Hydrol. Earth Syst. Sci., 22, 6005–6022, https://doi.org/10.5194/hess-22-6005-2018, 2018. </p><p>Kratzert, F., Klotz, D., Shalev, G., Klambauer, G., Hochreiter, S., and Nearing, G.: Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets, Hydrol. Earth Syst. Sci., 23, 5089–5110, https://doi.org/10.5194/hess-23-5089-2019, 2019a. </p><p>Kratzert, F., Klotz, D., Herrnegger, M., Sampson, A. K., Hochreiter, S., & Nearing, G. S.: Toward improved predictions in ungauged basins: Exploiting the power of machine learning. Water Resources Research, 55. https://doi.org/10.1029/2019WR026065, 2019b.</p>

scholarly journals Data-Driven Predictive Modeling of Neuronal Dynamics using Long Short-Term Memory

2019 ◽  
Author(s):  
Benjamin Plaster ◽  
Gautam Kumar
Keyword(s):  
Neural Network ◽  
Time Horizon ◽  
Short Term Memory ◽  
Temporal Dynamics ◽  
Data Driven ◽  
Brain Dynamics ◽  
Short Term ◽  
Term Memory ◽  
Brain Functions ◽  
Long Short Term Memory

Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of developing computationally efficient models of brain dynamics to use in designing control-theoretic neurostimulation strategies, we have developed a novel data-driven approach in a long short-term memory (LSTM) neural network architecture to predict the temporal dynamics of complex systems over an extended long time-horizon in future. In contrast to recent LSTM-based dynamical modeling approaches that make use of multi-layer perceptrons or linear combination layers as output layers, our architecture uses a single fully connected output layer and reversed-order sequence-to-sequence mapping to improve short time-horizon prediction accuracy and to make multi-timestep predictions of dynamical behaviors. We demonstrate the efficacy of our approach in reconstructing the regular spiking to bursting dynamics exhibited by an experimentally-validated 9-dimensional Hodgkin-Huxley model of hippocampal CA1 pyramidal neurons. Through simulations, we show that our LSTM neural network can predict the multi-time scale temporal dynamics underlying various spiking patterns with reasonable accuracy. Moreover, our results show that the predictions improve with increasing predictive time-horizon in the multi-timestep deep LSTM neural network.

Sign in / Sign up

Export Citation Format

Share Document