scholarly journals Investigation of Hyperparameter Setting of a Long Short-Term Memory Model Applied for Imputation of Missing Discharge Data of the Daihachiga River

Water ◽  
2022 ◽  
Vol 14 (2) ◽  
pp. 213
Author(s):  
Weilisi ◽  
Toshiharu Kojima

Missing observational data pose an unavoidable problem in the hydrological field. Deep learning technology has recently been developing rapidly, and has started to be applied in the hydrological field. Being one of the network architectures used in deep learning, Long Short-Term Memory (LSTM) has been applied largely in related research, such as flood forecasting and discharge prediction, and the performance of an LSTM model has been compared with other deep learning models. Although the tuning of hyperparameters, which influences the performance of an LSTM model, is necessary, no sufficient knowledge has been obtained. In this study, we tuned the hyperparameters of an LSTM model to investigate the influence on the model performance, and tried to obtain a more suitable hyperparameter combination for the imputation of missing discharge data of the Daihachiga River. A traditional method, linear regression with an accuracy of 0.903 in Nash–Sutcliffe Efficiency (NSE), was chosen as the comparison target of the accuracy. The results of most of the trainings that used the discharge data of both neighboring and estimation points had better accuracy than the regression. Imputation of 7 days of the missing period had a minimum value of 0.904 in NSE, and 1 day of the missing period had a lower quartile of 0.922 in NSE. Dropout value indicated a negative correlation with the accuracy. Setting dropout as 0 had the best accuracy, 0.917 in the lower quartile of NSE. When the missing period was 1 day and the number of hidden layers were more than 100, all the compared results had an accuracy of 0.907–0.959 in NSE. Consequently, the case, which used discharge data with backtracked time considering the missing period of 1 day and 7 days and discharge data of adjacent points as input data, indicated better accuracy than other input data combinations. Moreover, the following information is obtained for this LSTM model: 100 hidden layers are better, and dropout and recurrent dropout levels equaling 0 are also better. The obtained optimal combination of hyperparameters exceeded the accuracy of the traditional method of regression analysis.

Author(s):  
Claire Brenner ◽  
Jonathan Frame ◽  
Grey Nearing ◽  
Karsten Schulz

ZusammenfassungDie Verdunstung ist ein entscheidender Prozess im globalen Wasser‑, Energie- sowie Kohlenstoffkreislauf. Daten zur räumlich-zeitlichen Dynamik der Verdunstung sind daher von großer Bedeutung für Klimamodellierungen, zur Abschätzung der Auswirkungen der Klimakrise sowie nicht zuletzt für die Landwirtschaft.In dieser Arbeit wenden wir zwei Machine- und Deep Learning-Methoden für die Vorhersage der Verdunstung mit täglicher und halbstündlicher Auflösung für Standorte des FLUXNET-Datensatzes an. Das Long Short-Term Memory Netzwerk ist ein rekurrentes neuronales Netzwerk, welchen explizit Speichereffekte berücksichtigt und Zeitreihen der Eingangsgrößen analysiert (entsprechend physikalisch-basierten Wasserbilanzmodellen). Dem gegenüber gestellt werden Modellierungen mit XGBoost, einer Entscheidungsbaum-Methode, die in diesem Fall nur Informationen für den zu bestimmenden Zeitschritt erhält (entsprechend physikalisch-basierten Energiebilanzmodellen). Durch diesen Vergleich der beiden Modellansätze soll untersucht werden, inwieweit sich durch die Berücksichtigung von Speichereffekten Vorteile für die Modellierung ergeben.Die Analysen zeigen, dass beide Modellansätze gute Ergebnisse erzielen und im Vergleich zu einem ausgewerteten Referenzdatensatz eine höhere Modellgüte aufweisen. Vergleicht man beide Modelle, weist das LSTM im Mittel über alle 153 untersuchten Standorte eine bessere Übereinstimmung mit den Beobachtungen auf. Allerdings zeigt sich eine Abhängigkeit der Güte der Verdunstungsvorhersage von der Vegetationsklasse des Standorts; vor allem wärmere, trockene Standorte mit kurzer Vegetation werden durch das LSTM besser repräsentiert, wohingegen beispielsweise in Feuchtgebieten XGBoost eine bessere Übereinstimmung mit den Beobachtung liefert. Die Relevanz von Speichereffekten scheint daher zwischen Ökosystemen und Standorten zu variieren.Die präsentierten Ergebnisse unterstreichen das Potenzial von Methoden der künstlichen Intelligenz für die Beschreibung der Verdunstung.


2021 ◽  
Vol 13 (10) ◽  
pp. 1953
Author(s):  
Seyed Majid Azimi ◽  
Maximilian Kraus ◽  
Reza Bahmanyar ◽  
Peter Reinartz

In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the L1 and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, L1 yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research.


2021 ◽  
Vol 366 (1) ◽  
Author(s):  
Zhichao Wen ◽  
Shuhui Li ◽  
Lihua Li ◽  
Bowen Wu ◽  
Jianqiang Fu

Energies ◽  
2020 ◽  
Vol 13 (14) ◽  
pp. 3517 ◽  
Author(s):  
Anh Ngoc-Lan Huynh ◽  
Ravinesh C. Deo ◽  
Duc-Anh An-Vo ◽  
Mumtaz Ali ◽  
Nawin Raj ◽  
...  

This paper aims to develop the long short-term memory (LSTM) network modelling strategy based on deep learning principles, tailored for the very short-term, near-real-time global solar radiation (GSR) forecasting. To build the prescribed LSTM model, the partial autocorrelation function is applied to the high resolution, 1 min scaled solar radiation dataset that generates statistically significant lagged predictor variables describing the antecedent behaviour of GSR. The LSTM algorithm is adopted to capture the short- and the long-term dependencies within the GSR data series patterns to accurately predict the future GSR at 1, 5, 10, 15, and 30 min forecasting horizons. This objective model is benchmarked at a solar energy resource rich study site (Bac-Ninh, Vietnam) against the competing counterpart methods employing other deep learning, a statistical model, a single hidden layer and a machine learning-based model. The LSTM model generates satisfactory predictions at multiple-time step horizons, achieving a correlation coefficient exceeding 0.90, outperforming all of the counterparts. In accordance with robust statistical metrics and visual analysis of all tested data, the study ascertains the practicality of the proposed LSTM approach to generate reliable GSR forecasts. The Diebold–Mariano statistic test also shows LSTM outperforms the counterparts in most cases. The study confirms the practical utility of LSTM in renewable energy studies, and broadly in energy-monitoring devices tailored for other energy variables (e.g., hydro and wind energy).


2021 ◽  
pp. 016555152110065
Author(s):  
Rahma Alahmary ◽  
Hmood Al-Dossari

Sentiment analysis (SA) aims to extract users’ opinions automatically from their posts and comments. Almost all prior works have used machine learning algorithms. Recently, SA research has shown promising performance in using the deep learning approach. However, deep learning is greedy and requires large datasets to learn, so it takes more time for data annotation. In this research, we proposed a semiautomatic approach using Naïve Bayes (NB) to annotate a new dataset in order to reduce the human effort and time spent on the annotation process. We created a dataset for the purpose of training and testing the classifier by collecting Saudi dialect tweets. The dataset produced from the semiautomatic model was then used to train and test deep learning classifiers to perform Saudi dialect SA. The accuracy achieved by the NB classifier was 83%. The trained semiautomatic model was used to annotate the new dataset before it was fed into the deep learning classifiers. The three deep learning classifiers tested in this research were convolutional neural network (CNN), long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). Support vector machine (SVM) was used as the baseline for comparison. Overall, the performance of the deep learning classifiers exceeded that of SVM. The results showed that CNN reported the highest performance. On one hand, the performance of Bi-LSTM was higher than that of LSTM and SVM, and, on the other hand, the performance of LSTM was higher than that of SVM. The proposed semiautomatic annotation approach is usable and promising to increase speed and save time and effort in the annotation process.


Sign in / Sign up

Export Citation Format

Share Document