Long Short Term Memory and Non-linear Autoregressive Models for Prayer Time Prediction

2021 ◽  
Author(s):  
Adamu Lawan ◽  
Muhammad Lawan ◽  
Ahmad Dikko Umar ◽  
Abba Mukhtar Bala
Keyword(s):  
Short Term Memory ◽  
Autoregressive Models ◽  
Short Term ◽  
Term Memory ◽  
Time Prediction ◽  
Non Linear ◽  
Long Short Term Memory

scholarly journals Using long short-term memory networks for river flow prediction

2020 ◽  
Vol 51 (6) ◽  
pp. 1358-1376
Author(s):  
Wei Xu ◽  
Yanan Jiang ◽  
Xiaoli Zhang ◽  
Yi Li ◽  
Run Zhang ◽  
...  
Keyword(s):  
River Flow ◽  
Short Term Memory ◽  
River Basins ◽  
Short Term ◽  
Term Memory ◽  
Learning Efficiency ◽  
Flow Prediction ◽  
Non Linear ◽  
Long Short Term Memory ◽  
The Impact

Abstract Deep learning has made significant advances in methodologies and practical applications in recent years. However, there is a lack of understanding on how the long short-term memory (LSTM) networks perform in river flow prediction. This paper assesses the performance of LSTM networks to understand the impact of network structures and parameters on river flow predictions. Two river basins with different characteristics, i.e., Hun river and Upper Yangtze river basins, are used as case studies for the 10-day average flow predictions and the daily flow predictions, respectively. The use of the fully connected layer with the activation function before the LSTM cell layer can substantially reduce learning efficiency. On the contrary, non-linear transformation following the LSTM cells is required to improve learning efficiency due to the different magnitudes of precipitation and flow. The batch size and the number of LSTM cells are sensitive parameters and should be carefully tuned to achieve a balance between learning efficiency and stability. Compared with several hydrological models, the LSTM network achieves good performance in terms of three evaluation criteria, i.e., coefficient of determination, Nash–Sutcliffe Efficiency and relative error, which demonstrates its powerful capacity in learning non-linear and complex processes in hydrological modelling.

scholarly journals Groundwater level forecasting with artificial neural networks: a comparison of long short-term memory (LSTM), convolutional neural networks (CNNs), and non-linear autoregressive networks with exogenous input (NARX)

2021 ◽  
Vol 25 (3) ◽  
pp. 1671-1687
Author(s):  
Andreas Wunsch ◽  
Tanja Liesch ◽  
Stefan Broda
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Convolutional Neural Networks ◽  
Groundwater Level ◽  
Short Term Memory ◽  
State Of The Art ◽  
Short Term ◽  
Term Memory ◽  
Non Linear ◽  
Long Short Term Memory

Abstract. It is now well established to use shallow artificial neural networks (ANNs) to obtain accurate and reliable groundwater level forecasts, which are an important tool for sustainable groundwater management. However, we observe an increasing shift from conventional shallow ANNs to state-of-the-art deep-learning (DL) techniques, but a direct comparison of the performance is often lacking. Although they have already clearly proven their suitability, shallow recurrent networks frequently seem to be excluded from the study design due to the euphoria about new DL techniques and its successes in various disciplines. Therefore, we aim to provide an overview on the predictive ability in terms of groundwater levels of shallow conventional recurrent ANNs, namely non-linear autoregressive networks with exogenous input (NARX) and popular state-of-the-art DL techniques such as long short-term memory (LSTM) and convolutional neural networks (CNNs). We compare the performance on both sequence-to-value (seq2val) and sequence-to-sequence (seq2seq) forecasting on a 4-year period while using only few, widely available and easy to measure meteorological input parameters, which makes our approach widely applicable. Further, we also investigate the data dependency in terms of time series length of the different ANN architectures. For seq2val forecasts, NARX models on average perform best; however, CNNs are much faster and only slightly worse in terms of accuracy. For seq2seq forecasts, mostly NARX outperform both DL models and even almost reach the speed of CNNs. However, NARX are the least robust against initialization effects, which nevertheless can be handled easily using ensemble forecasting. We showed that shallow neural networks, such as NARX, should not be neglected in comparison to DL techniques especially when only small amounts of training data are available, where they can clearly outperform LSTMs and CNNs; however, LSTMs and CNNs might perform substantially better with a larger dataset, where DL really can demonstrate its strengths, which is rarely available in the groundwater domain though.

Toward Downscaling Storage-Discharge Dynamics: Training Long Short-Term Memory (LSTM) model for Simulating Nonlinear Storage-Discharge Relations at The Rum River Watershed, MN

2021 ◽  
Author(s):  
Pai-Feng Teng ◽  
John Nieber
Keyword(s):  
Short Term Memory ◽  
Water Storage ◽  
Rainfall Runoff ◽  
Total Water ◽  
Short Term ◽  
Term Memory ◽  
Non Linear ◽  
Long Short Term Memory ◽  
The Relationship ◽  
Hydrological Variables

<p>Flooding is one of the most financially devastating natural hazards in the world. Studying storage-discharge relations can have the potential to improve existing flood forecasting systems, which are based on rainfall-runoff models. This presentation will assess the non-linear relation between daily water storage (ΔS) and discharge (Q) simulated by physical-based hydrological models at the Rum River Watershed, a HUC8 watershed in Minnesota, between 1995-2015, by training Long Short-Term Memory (LSTM) networks and other machine learning (ML) algorithms. Currently, linear regression models do not adequately represent the relationship between the simulated total ΔS and total Q at the HUC-8 watershed (R<sup>2</sup> = 0.3667). Since ML algorithms have been used for predicting the outputs that represent arbitrary non-linear functions between predictors and predictands, they will be used for improving the accuracy of the non-linear relation of the storage-discharge dynamics. This research will mainly use LSTM networks, the time-series deep learning neural network that has already been used for predicting rainfall-runoff relations. The LSTM network will be trained to evaluate the storage-discharge relationship by comparing two sets of non-linear hydrological variables simulated by the semi-distributed Hydrological Simulated Program-Fortran (HSPF): the relationship between the simulated discharges and input hydrological variables at selected HUC-8 watersheds, including air temperatures, cloud covers, dew points, potential evapotranspiration, precipitations, solar radiations, wind speeds, and total water storage, and the dynamics between simulated discharge and input variables that do not include the total water storage. The result of this research will lay the foundation for assessing the accuracy of downscaled storage-discharge dynamics by applying similar methods to evaluate the storage-discharge dynamics at small-scaled, HUC-12 watersheds. Furthermore, its results have the potentials for us to evaluate whether downscaling of storage-discharge dynamics at the HUC-12 watershed can improve the accuracy of predicting discharge by comparing the result from the HUC-8 and the HUC-12 watersheds.</p>

scholarly journals Towards Attention-Based Convolutional Long Short-Term Memory for Travel Time Prediction of Bus Journeys

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3354 ◽  
Author(s):  
Jianqing Wu ◽  
Qiang Wu ◽  
Jun Shen ◽  
Chen Cai
Keyword(s):  
Information Systems ◽  
Travel Time ◽  
Short Term Memory ◽  
Transportation Systems ◽  
Series Data ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Time Prediction ◽  
Long Short Term Memory

Travel time prediction is critical for advanced traveler information systems (ATISs), which provides valuable information for enhancing the efficiency and effectiveness of the urban transportation systems. However, in the area of bus trips, existing studies have focused on directly using the structured data to predict travel time for a single bus trip. For state-of-the-art public transportation information systems, a bus journey generally has multiple bus trips. Additionally, due to the lack of study on data fusion, it is even inadequate for the development of underlying intelligent transportation systems. In this paper, we propose a novel framework for a hybrid data-driven travel time prediction model for bus journeys based on open data. We explore a convolutional long short-term memory (ConvLSTM) model with a self-attention mechanism that accurately predicts the running time of each segment of the trips and the waiting time at each station. The model is more robust to capture long-range dependence in time series data as well.

scholarly journals Sales forecasting using multivariate long short term memory network models

2019 ◽  
Author(s):  
Suleka Helmini ◽  
Nadheesh Jihan ◽  
Malith Jayasinghe ◽  
Srinath Perera
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Short Term Memory ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Forecasting Models ◽  
Non Linear ◽  
Long Short Term Memory ◽  
Machine Learning Models

In the retail domain, estimating the sales before actual sales become known plays a key role in maintaining a successful business. This is due to the fact that most crucial decisions are bound to be based on these forecasts. Statistical sales forecasting models like ARIMA (Auto-Regressive Integrated Moving Average), can be identified as one of the most traditional and commonly used forecasting methodologies. Even though these models are capable of producing satisfactory forecasts for linear time series data they are not suitable for analyzing non-linear data. Therefore, machine learning models (such as Random Forest Regression, XGBoost) have been employed frequently as they were able to achieve better results using non-linear data. The recent research shows that deep learning models (e.g. recurrent neural networks) can provide higher accuracy in predictions compared to machine learning models due to their ability to persist information and identify temporal relationships. In this paper, we adopt a special variant of Long Short Term Memory (LSTM) network called LSTM model with peephole connections for sales prediction. We first build our model using historical features for sales forecasting. We compare the results of this initial LSTM model with multiple machine learning models, namely, the Extreme Gradient Boosting model (XGB) and Random Forest Regressor model(RFR). We further improve the prediction accuracy of the initial model by incorporating features that describe the future that is known to us in the current moment, an approach that has not been explored in previous state-of-the-art LSTM based forecasting models. The initial LSTM model we develop outperforms the machine learning models achieving 12% - 14% improvement whereas the improved LSTM model achieves 11\% - 13\% improvement compared to the improved machine learning models. Furthermore, we also show that our improved LSTM model can obtain a 20% - 21% improvement compared to the initial LSTM model, achieving significant improvement.

scholarly journals Application of Long Short Term Memory Networks for Long- and Short-term Bus Travel Time Prediction

2021 ◽  
Author(s):  
Osama Osman ◽  
Hesham Rakha ◽  
Archak Mittal
Keyword(s):  
Travel Time ◽  
Traffic Control ◽  
Short Term Memory ◽  
Temporal Correlation ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Time Prediction ◽  
Long Short Term Memory ◽  
Control Devices

This study introduces a comparative analysis of two deep learning (multilayer perceptron neural networks (MLP-NN) and the long short term memory networks (LSTMN)) models for transit travel time prediction. The two models were trained and tested using one-year worth of data for a bus route in Blacksburg, Virginia. In this study, the travel time was predicted between each two successive stations to all the model to be extended to include bus dwell times. Additionally, two additional models were developed for each category (MLP of LSTM): one for only segments including controlled intersections (controlled segments) and another for segments with no control devices along them (uncontrolled segments). The results show that the LSTM models outperform the MLP models with a RMSE of 17.69 sec compared to 18.81 sec. When splitting the data into controlled and uncontrolled segments, the RMSE values reduced to 17.33 sec for the controlled segments and 4.28 sec for the uncontrolled segments when applying the LSTM model. Whereas, the RMSE values were 19.39 sec for the controlled segments and 4.67 sec for the uncontrolled segments when applying the MLP model. These results demonstrate that the uncertainty in traffic conditions introduced by traffic control devices has a significant impact on travel time predictions. Nonetheless, the results demonstrate that the LSTMN is a promising tool that can has the ability to account for the temporal correlation within the data. The developed models are also promising tools for reasonable travel time predictions in transit applications.

scholarly journals Sales forecasting using multivariate long short term memory network models

2019 ◽  
Author(s):  
Suleka Helmini ◽  
Nadheesh Jihan ◽  
Malith Jayasinghe ◽  
Srinath Perera
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Short Term Memory ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Forecasting Models ◽  
Non Linear ◽  
Long Short Term Memory ◽  
Machine Learning Models

In the retail domain, estimating the sales before actual sales become known plays a key role in maintaining a successful business. This is due to the fact that most crucial decisions are bound to be based on these forecasts. Statistical sales forecasting models like ARIMA (Auto-Regressive Integrated Moving Average), can be identified as one of the most traditional and commonly used forecasting methodologies. Even though these models are capable of producing satisfactory forecasts for linear time series data they are not suitable for analyzing non-linear data. Therefore, machine learning models (such as Random Forest Regression, XGBoost) have been employed frequently as they were able to achieve better results using non-linear data. The recent research shows that deep learning models (e.g. recurrent neural networks) can provide higher accuracy in predictions compared to machine learning models due to their ability to persist information and identify temporal relationships. In this paper, we adopt a special variant of Long Short Term Memory (LSTM) network called LSTM model with peephole connections for sales prediction. We first build our model using historical features for sales forecasting. We compare the results of this initial LSTM model with multiple machine learning models, namely, the Extreme Gradient Boosting model (XGB) and Random Forest Regressor model(RFR). We further improve the prediction accuracy of the initial model by incorporating features that describe the future that is known to us in the current moment, an approach that has not been explored in previous state-of-the-art LSTM based forecasting models. The initial LSTM model we develop outperforms the machine learning models achieving 12% - 14% improvement whereas the improved LSTM model achieves 11\% - 13\% improvement compared to the improved machine learning models. Furthermore, we also show that our improved LSTM model can obtain a 20% - 21% improvement compared to the initial LSTM model, achieving significant improvement.

scholarly journals A Novel Cellular Automata Classifier for COVID-19 Prediction

2020 ◽  
Author(s):  
Kiran Sree Pokkuluri ◽  
SSSN Usha Devi Nedunuri
Keyword(s):  
Cellular Automata ◽  
Short Term Memory ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Hybrid Cellular Automata ◽  
The World ◽  
Non Linear ◽  
Long Short Term Memory ◽  
Linear Hybrid Cellular Automata

Introduction: China has witnessed a new virus Corona,which is named COVID-19. It has become the world’s most concern as this virus has spread over the worldat a higher speed;the world has witnessed more than one lakh cases and one thousand deaths in a span of few days. Methods: We have developed a preliminary classifier with non-linear hybrid cellular automata, which is trained and tested to predict the effect of COVID-19 in terms of deaths, the number of people affected, the number of people being could be recovered, etc. This indirectly predicts the trend of this epidemic in India. We have collected the datasets from Kaggle and other standard websites. Results: The proposed classifier, hybrid non-linear cellular automata (HNLCA), was trained with 23,078 datasets and tested with 6785 datasets. HNLCA is compared with conventional methods of long short-term memory, AdaBoost, support vector machine, regression, and SVR and has reported an accuracy of 78.8%, which is better compared with the cited literature. This classifier can also predict the rate at which this virus spreads, transmission within the boundary, and of the boundary, etc.

Sign in / Sign up

Export Citation Format

Share Document