scholarly journals How can regime characteristics of catchments help in training of local and regional LSTM-based runoff models?

2021 ◽  
Author(s):  
Reyhaneh Hashemi ◽  
Pierre Brigode ◽  
Pierre-André Garambois ◽  
Pierre Javelle
Keyword(s):  
Significant Contribution ◽  
Short Term Memory ◽  
Linear Time ◽  
Climatic Conditions ◽  
Regional Models ◽  
Unseen Data ◽  
Non Linear ◽  
Long Short Term Memory ◽  
Dependency Structures

Abstract. In the field of Deep Learning, the long short-term memory (LSTM) networks lie in the category of recurrent neural network (RNN) architectures. The distinctive capability of the LSTM is learning non linear long term dependency structures. This makes the LSTM a good candidate for prediction tasks in non linear time dependent systems such as prediction of runoff in a catchment. In this study, we use a large sample of 740 gauged catchments with very diverse hydro-geo-climatic conditions across France. We present a regime classification based on three hydro-climatic indices to identify and classify catchments with similar hydrological behaviors. We do this because we aim to investigate how regime derived information can be used in training LSTM-based runoff models. The LSTM-based models that we investigate include local models trained on individual catchments as well as regional models trained on a group of catchments. In local training, for each regime, we identify the optimal lookback, i.e. the length of the sequence of past forcing data that the LSTM needs to work through. We then use this length in training regional models that differ in two aspects: 1) hydrological homogeneity of the catchments used in their training, 2) configuration of the static attributes used in their inputs. We examine how each of these aspects contributes to learning of the LSTM in regional training. At every step of this study, we benchmark performances of the LSTM against a conceptual model (GR4J) on both train and unseen data. We show that the optimal lookback is regime dependent and homogeneity of the train catchments in regional training has a more significant contribution to learning of the LSTM than the number of the train catchments.

scholarly journals Long Short Term Memory Recurrent Network for Standard and Poor’s 500 Index Modelling

2018 ◽  
Vol 7 (4.15) ◽  
pp. 25 ◽  
Author(s):  
Said Jadid Abdulkadir ◽  
Hitham Alhussian ◽  
Muhammad Nazmi ◽  
Asim A Elsheikh
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Short Term Memory ◽  
Series Data ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Forecasting time-series data are imperative especially when planning is required through modelling using uncertain knowledge of future events. Recurrent neural network models have been applied in the industry and outperform standard artificial neural networks in forecasting, but fail in long term time-series forecasting due to the vanishing gradient problem. This study offers a robust solution that can be implemented for long-term forecasting using a special architecture of recurrent neural network known as Long Short Term Memory (LSTM) model to overcome the vanishing gradient problem. LSTM is specially designed to avoid the long-term dependency problem as their default behavior. Empirical analysis is performed using quantitative forecasting metrics and comparative model performance on the forecasted outputs. An evaluation analysis is performed to validate that the LSTM model provides better forecasted outputs on Standard & Poor’s 500 Index (S&P 500) in terms of error metrics as compared to other forecasting models.  

scholarly journals Deep Neural Network and Long Short-Term Memory for Electric Power Load Forecasting

2020 ◽  
Vol 10 (18) ◽  
pp. 6489
Author(s):  
Namrye Son ◽  
Seunghak Yang ◽  
Jeongseung Na
Keyword(s):  
Power Consumption ◽  
Short Term Memory ◽  
Load Forecasting ◽  
Power Demand ◽  
Short Term ◽  
Power Load ◽  
Short And Long Term ◽  
Long Short Term Memory ◽  
Electric Power Load

Forecasting domestic and foreign power demand is crucial for planning the operation and expansion of facilities. Power demand patterns are very complex owing to energy market deregulation. Therefore, developing an appropriate power forecasting model for an electrical grid is challenging. In particular, when consumers use power irregularly, the utility cannot accurately predict short- and long-term power consumption. Utilities that experience short- and long-term power demands cannot operate power supplies reliably; in worst-case scenarios, blackouts occur. Therefore, the utility must predict the power demands by analyzing the customers’ power consumption patterns for power supply stabilization. For this, a medium- and long-term power forecasting is proposed. The electricity demand forecast was divided into medium-term and long-term load forecast for customers with different power consumption patterns. Among various deep learning methods, deep neural networks (DNNs) and long short-term memory (LSTM) were employed for the time series prediction. The DNN and LSTM performances were compared to verify the proposed model. The two models were tested, and the results were examined with the accuracies of the six most commonly used evaluation measures in the medium- and long-term electric power load forecasting. The DNN outperformed the LSTM, regardless of the customer’s power pattern.

scholarly journals Learning Robot Speech Models to Predict Speech Acts in HRI

2016 ◽  
Vol 9 (1) ◽  
pp. 295-306
Author(s):  
Ankuj Arora ◽  
Humbert Fiorino ◽  
Damien Pellier ◽  
Sylvie Pesty
Keyword(s):  
Speech Acts ◽  
Short Term Memory ◽  
The State ◽  
Short Term ◽  
Long Run ◽  
Action Sequences ◽  
Long Short Term Memory ◽  
Max Sat ◽  
Speech Model

Abstract In order to be acceptable and able to “camouflage” into their physio-social context in the long run, robots need to be not just functional, but autonomously psycho-affective as well. This motivates a long term necessity of introducing behavioral autonomy in robots, so they can autonomously communicate with humans without the need of “wizard” intervention. This paper proposes a technique to learn robot speech models from human-robot dialog exchanges. It views the entire exchange in the Automated Planning (AP) paradigm, representing the dialog sequences (speech acts) in the form of action sequences that modify the state of the world upon execution, gradually propelling the state to a desired goal. We then exploit intra-action and inter-action dependencies, encoding them in the form of constraints. We attempt to satisfy these constraints using aweighted maximum satisfiability model known as MAX-SAT, and convert the solution into a speech model. This model could have many uses, such as planning of fresh dialogs. In this study, the learnt model is used to predict speech acts in the dialog sequences using the sequence labeling (predicting future acts based on previously seen ones) capabilities of the LSTM (Long Short Term Memory) class of recurrent neural networks. Encouraging empirical results demonstrate the utility of this learnt model and its long term potential to facilitate autonomous behavioral planning of robots, an aspect to be explored in future works.

scholarly journals Using long short-term memory networks for river flow prediction

2020 ◽  
Vol 51 (6) ◽  
pp. 1358-1376
Author(s):  
Wei Xu ◽  
Yanan Jiang ◽  
Xiaoli Zhang ◽  
Yi Li ◽  
Run Zhang ◽  
...  
Keyword(s):  
River Flow ◽  
Short Term Memory ◽  
River Basins ◽  
Short Term ◽  
Term Memory ◽  
Learning Efficiency ◽  
Flow Prediction ◽  
Non Linear ◽  
Long Short Term Memory ◽  
The Impact

Abstract Deep learning has made significant advances in methodologies and practical applications in recent years. However, there is a lack of understanding on how the long short-term memory (LSTM) networks perform in river flow prediction. This paper assesses the performance of LSTM networks to understand the impact of network structures and parameters on river flow predictions. Two river basins with different characteristics, i.e., Hun river and Upper Yangtze river basins, are used as case studies for the 10-day average flow predictions and the daily flow predictions, respectively. The use of the fully connected layer with the activation function before the LSTM cell layer can substantially reduce learning efficiency. On the contrary, non-linear transformation following the LSTM cells is required to improve learning efficiency due to the different magnitudes of precipitation and flow. The batch size and the number of LSTM cells are sensitive parameters and should be carefully tuned to achieve a balance between learning efficiency and stability. Compared with several hydrological models, the LSTM network achieves good performance in terms of three evaluation criteria, i.e., coefficient of determination, Nash–Sutcliffe Efficiency and relative error, which demonstrates its powerful capacity in learning non-linear and complex processes in hydrological modelling.

scholarly journals An LSTM-Based Method with Attention Mechanism for Travel Time Prediction

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 861 ◽  
Author(s):  
Xiangdong Ran ◽  
Zhiguang Shan ◽  
Yufei Fang ◽  
Chuang Lin
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Traffic Prediction ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Departure Time ◽  
Long Short Term Memory

Traffic prediction is based on modeling the complex non-linear spatiotemporal traffic dynamics in road network. In recent years, Long Short-Term Memory has been applied to traffic prediction, achieving better performance. The existing Long Short-Term Memory methods for traffic prediction have two drawbacks: they do not use the departure time through the links for traffic prediction, and the way of modeling long-term dependence in time series is not direct in terms of traffic prediction. Attention mechanism is implemented by constructing a neural network according to its task and has recently demonstrated success in a wide range of tasks. In this paper, we propose an Long Short-Term Memory-based method with attention mechanism for travel time prediction. We present the proposed model in a tree structure. The proposed model substitutes a tree structure with attention mechanism for the unfold way of standard Long Short-Term Memory to construct the depth of Long Short-Term Memory and modeling long-term dependence. The attention mechanism is over the output layer of each Long Short-Term Memory unit. The departure time is used as the aspect of the attention mechanism and the attention mechanism integrates departure time into the proposed model. We use AdaGrad method for training the proposed model. Based on the datasets provided by Highways England, the experimental results show that the proposed model can achieve better accuracy than the Long Short-Term Memory and other baseline methods. The case study suggests that the departure time is effectively employed by using attention mechanism.

scholarly journals Long Short-Term Memory with Dynamic Skip Connections

2019 ◽  
Vol 33 ◽  
pp. 6481-6488 ◽  
Author(s):  
Tao Gui ◽  
Qi Zhang ◽  
Lujun Zhao ◽  
Yaosong Lin ◽  
Minlong Peng ◽  
...  
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Training Data ◽  
Sequential Data ◽  
Short Term ◽  
Term Memory ◽  
Transition Functions ◽  
Proposed Model ◽  
Long Short Term Memory

In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length. However, LSTM can still experience difficulty in capturing long-term dependencies. In this work, we tried to alleviate this problem by introducing a dynamic skip connection, which can learn to directly connect two dependent words. Since there is no dependency information in the training data, we propose a novel reinforcement learning-based method to model the dependency relationship and connect dependent words. The proposed model computes the recurrent transition functions based on the skip connections, which provides a dynamic skipping advantage over RNNs that always tackle entire sentences sequentially. Our experimental results on three natural language processing tasks demonstrate that the proposed method can achieve better performance than existing methods. In the number prediction experiment, the proposed model outperformed LSTM with respect to accuracy by nearly 20%.

Sign in / Sign up

Export Citation Format

Share Document