Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation

Considering the high random and non-static property of the rainfall-runoff process, lots of models are being developed in order to learn about such a complex phenomenon. Recently, Machine learning techniques such as the Artificial Neural Network (ANN) and other networks have been extensively used by hydrologists for rainfall-runoff modelling as well as for other fields of hydrology. However, deep learning methods such as the state-of-the-art for LSTM networks are little studied in hydrological sequence time-series predictions. We deployed ANN and LSTM network models for simulating the rainfall-runoff process based on flood events from 1971 to 2013 in Fen River basin monitored through 14 rainfall stations and one hydrologic station in the catchment. The experimental data were from 98 rainfall-runoff events in this period. In between 86 rainfall-runoff events were used as training set, and the rest were used as test set. The results show that the two networks are all suitable for rainfall-runoff models and better than conceptual and physical based models. LSTM models outperform the ANN models with the values of R 2 and N S E beyond 0.9, respectively. Considering different lead time modelling the LSTM model is also more stable than ANN model holding better simulation performance. The special units of forget gate makes LSTM model better simulation and more intelligent than ANN model. In this study, we want to propose new data-driven methods for flood forecasting.

Download Full-text

Deep learning convolutional neural network in rainfall–runoff modelling

Journal of Hydroinformatics ◽

10.2166/hydro.2020.095 ◽

2020 ◽

Vol 22 (3) ◽

pp. 541-561 ◽

Cited By ~ 4

Author(s):

Song Pham Van ◽

Hoang Minh Le ◽

Dat Vi Thanh ◽

Thanh Duc Dang ◽

Ho Huu Loc ◽

...

Keyword(s):

Neural Network ◽

Time Series ◽

Deep Learning ◽

Convolutional Neural Network ◽

Short Term Memory ◽

Water Cycle ◽

Machine Learning Techniques ◽

Analytical Models ◽

Rainfall Runoff ◽

Convolutional Network

Abstract Rainfall–runoff modelling is complicated due to numerous complex interactions and feedback in the water cycle among precipitation and evapotranspiration processes, and also geophysical characteristics. Consequently, the lack of geophysical characteristics such as soil properties leads to difficulties in developing physical and analytical models when traditional statistical methods cannot simulate rainfall–runoff accurately. Machine learning techniques with data-driven methods, which can capture the nonlinear relationship between prediction and predictors, have been rapidly developed in the last decades and have many applications in the field of water resources. This study attempts to develop a novel 1D convolutional neural network (CNN), a deep learning technique, with a ReLU activation function for rainfall–runoff modelling. The modelling paradigm includes applying two convolutional filters in parallel to separate time series, which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. The developed modelling framework is evaluated with measured data at Chau Doc and Can Tho hydro-meteorological stations in the Vietnamese Mekong Delta. The proposed model results are compared with simulations of long short-term memory (LSTM) and traditional models. Both CNN and LSTM have better performance than the traditional models, and the statistical performance of the CNN model is slightly better than the LSTM results. We demonstrate that the convolutional network is suitable for regression-type problems and can effectively learn dependencies in and between the series without the need for a long historical time series, is a time-efficient and easy to implement alternative to recurrent-type networks and tends to outperform linear and recurrent models.

Download Full-text

Comprehensive comparison of artificial neural networks and long short-term memory networks for rainfall-runoff simulation

Physics and Chemistry of the Earth Parts A/B/C ◽

10.1016/j.pce.2021.103026 ◽

2021 ◽

pp. 103026

Author(s):

Ganquan Mao ◽

Meng Wang ◽

Junguo Liu ◽

Zifeng Wang ◽

Kai Wang ◽

...

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

Short Term Memory ◽

Rainfall Runoff ◽

Short Term ◽

Runoff Simulation ◽

Term Memory ◽

Comprehensive Comparison ◽

Long Short Term Memory ◽

Artificial Neural

Download Full-text

AOHDL: Archimedes Optimization Based Hybrid Deep Learning Model for Soybean Plant Disease Classification

10.21203/rs.3.rs-302084/v1 ◽

2021 ◽

Author(s):

J. Annrose ◽

N. Herald Anantha Rufus ◽

C. R. Edwin Selva Rex ◽

D. Godwin Immanuel

Keyword(s):

Deep Learning ◽

Optimization Algorithm ◽

Short Term Memory ◽

Learning Model ◽

Disease Classification ◽

Training Data ◽

Machine Learning Techniques ◽

Soybean Plant ◽

Lower Accuracy ◽

Deep Learning Model

Abstract Bean which is botanically called Phaseolus vulgaris L belongs to the Fabaceae family.During bean disease identification, unnecessary economical losses occur due to the delay of the treatment period, incorrect treatment, and lack of knowledge. The existing deep learning and machine learning techniques met few issues such as high computational complexity, higher cost associated with the training data, more execution time, noise, feature dimensionality, lower accuracy, low speed, etc. To tackle these problems, we have proposed a hybrid deep learning model with an Archimedes optimization algorithm (HDL-AOA) for bean disease classification. In this work, there are five bean classes of which one is a healthy class whereas the remaining four classes indicate different diseases such as Bean halo blight, Pythium diseases, Rhizoctonia root rot, and Anthracnose abnormalities acquired from the Soybean (Large) Data Set.The hybrid deep learning technique is the combination of wavelet packet decomposition (WPD) and long short term memory (LSTM). Initially, the WPD decomposes the input images into four sub-series. For these sub-series, four LSTM networks were developed. During bean disease classification, an Archimedes optimization algorithm (AOA) enhances the classification accuracy for multiple single LSTM networks. MATLAB software implements the HDL-AOA model for bean disease classification. The proposed model accomplishes lower MAPE than other exiting methods. Finally, the proposed HDL-AOA model outperforms excellent classification results using different evaluation measures such as accuracy, specificity, sensitivity, precision, recall, and F-score.

Download Full-text

Comparative analysis of Kernel-based versus BFGS-ANN and deep learning methods in monthly reference evaporation estimation

10.5194/hess-2020-224 ◽

2020 ◽

Author(s):

Mohammad Taghi Sattari ◽

Halit Apaydin ◽

Shahab Shamshirband ◽

Amir Mosavi

Keyword(s):

Deep Learning ◽

Minimum Temperature ◽

Meteorological Parameters ◽

Short Term Memory ◽

Sunshine Duration ◽

Gaussian Process Regression ◽

Support Vector ◽

Ann Model ◽

Learning Methods ◽

Average Maximum

Abstract. Proper estimation of the reference evapotranspiration (ET0) amount is an indispensable matter for agricultural water management in the efficient use of water. The aim of study is to estimate the amount of ET0 with a different machine and deep learning methods by using minimum meteorological parameters in the Corum region which is an arid and semi-arid climate with an important agricultural center of Turkey. In this context, meteorological variables of average, maximum and minimum temperature, sunshine duration, wind speed, average, maximum, and minimum relative humidity are used as input data monthly. Two different kernel-based (Gaussian Process Regression (GPR) and Support Vector Regression (SVR)) methods, BFGS-ANN and Long short-term memory models were used to estimate ET0 amounts in 10 different combinations. According to the results obtained, all four methods used predicted ET0 amounts in acceptable accuracy and error levels. BFGS-ANN model showed higher success than the others. In kernel-based GPR and SVR methods, Pearson VII function-based universal kernel was the most successful kernel function. Besides, the scenario that is related to temperature in all scenarios used, including average temperature, maximum and minimum temperature, and sunshine duration gave the best results. The second-best scenario was the one that covers only the sunshine duration. In this case, the ANN (BFGS-ANN) model, which is optimized with the BFGS method that uses only the sunshine duration, can be estimated with the 0.971 correlation coefficient of ET0 without the need for other meteorological parameters.

Download Full-text

Soybean Plant Disease Classification using Archimedes Optimization Algorithm based Hybrid Deep Learning Model

10.21203/rs.3.rs-281525/v1 ◽

2021 ◽

Author(s):

J. Annrose ◽

N. Herald Anantha Rufus ◽

C. R. Edwin Selva Rex ◽

D. Godwin Immanuel

Keyword(s):

Deep Learning ◽

Optimization Algorithm ◽

Short Term Memory ◽

Learning Model ◽

Disease Classification ◽

Training Data ◽

Machine Learning Techniques ◽

Soybean Plant ◽

Lower Accuracy ◽

Deep Learning Model

Download Full-text

Prediction of the optimal FSW process parameters for joints using machine learning techniques

Materials Testing ◽

10.1515/mt-2021-0058 ◽

2021 ◽

Vol 63 (12) ◽

pp. 1104-1111

Author(s):

Furkan Sarsilmaz ◽

Gürkan Kavuran

Keyword(s):

Mechanical Properties ◽

Tensile Strength ◽

Machine Learning Techniques ◽

Support Vector ◽

Ann Model ◽

Friction Stir ◽

Learning Techniques ◽

Svm Model ◽

Artificial Neural Network Ann ◽

Fsw Parameters

Abstract In this work, a couple of dissimilar AA2024/AA7075 plates were experimentally welded for the purpose of considering the effect of friction-stir welding (FSW) parameters on mechanical properties. First, the main mechanical properties such as ultimate tensile strength (UTS) and hardness of welded joints were determined experimentally. Secondly, these data were evaluated through modeling and the optimization of the FSW process as well as an optimal parametric combination to affirm tensile strength and hardness using a support vector machine (SVM) and an artificial neural network (ANN). In this study, a new ANN model, including the Nelder-Mead algorithm, was first used and compared with the SVM model in the FSW process. It was concluded that the ANN approach works better than SVM techniques. The validity and accuracy of the proposed method were proved by simulation studies.

Download Full-text

Development of Neural Network Models for a Crude Oil Distillation Column

Jurnal Teknologi ◽

10.11113/jt.v38.509 ◽

2012 ◽

Author(s):

Khairiyah Mohd. Yusof ◽

Fakhri Karray ◽

Peter L. Douglas

Keyword(s):

Neural Network ◽

Steady State ◽

Crude Oil ◽

Distillation Column ◽

Network Models ◽

Ann Model ◽

Neural Network Models ◽

Ann Models ◽

Artificial Neural Network Ann ◽

Crude Oil Distillation

This paper discusses the development of artificial neural network (ANN) models for a crude oil distillation column. Since the model is developed for real time optimisation (RTO) applications they are steady state, multivariable models. Training and testing data used to develop the models were generated from a reconciled steady-state model simulated in a process simulator. The radial basis function networks (RBFN), a type of feedforward ANN model, were able to model the crude tower very well, with the root mean square error for the prediction of each variable less than 1%. Grouping related output variables in a network model was found to give better predictions than lumping all the variables in a single model; this also allowed the overall complex, multivariable model to be simplified into smaller models that are more manageable. In addition, the RBFN models were also able to satisfactorily perform range and dimensional extrapolation, which is necessary for models that are used in RTO.

Download Full-text

Deploying Machine and Deep Learning Models for Efficient Data-Augmented Detection of COVID-19 Infections

Viruses ◽

10.3390/v12070769 ◽

2020 ◽

Vol 12 (7) ◽

pp. 769 ◽

Cited By ~ 12

Author(s):

Ahmed Sedik ◽

Abdullah M Iliyasu ◽

Basma Abd El-Rahiem ◽

Mohammed E. Abdel Samea ◽

Asmaa Abdel-Raheem ◽

...

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Short Term Memory ◽

Machine Learning Techniques ◽

Detection Accuracy ◽

Testing Time ◽

Learning Models ◽

Accurate Identification ◽

Infected People ◽

Corona Virus

This generation faces existential threats because of the global assault of the novel Corona virus 2019 (i.e., COVID-19). With more than thirteen million infected and nearly 600000 fatalities in 188 countries/regions, COVID-19 is the worst calamity since the World War II. These misfortunes are traced to various reasons, including late detection of latent or asymptomatic carriers, migration, and inadequate isolation of infected people. This makes detection, containment, and mitigation global priorities to contain exposure via quarantine, lockdowns, work/stay at home, and social distancing that are focused on “flattening the curve”. While medical and healthcare givers are at the frontline in the battle against COVID-19, it is a crusade for all of humanity. Meanwhile, machine and deep learning models have been revolutionary across numerous domains and applications whose potency have been exploited to birth numerous state-of-the-art technologies utilised in disease detection, diagnoses, and treatment. Despite these potentials, machine and, particularly, deep learning models are data sensitive, because their effectiveness depends on availability and reliability of data. The unavailability of such data hinders efforts of engineers and computer scientists to fully contribute to the ongoing assault against COVID-19. Faced with a calamity on one side and absence of reliable data on the other, this study presents two data-augmentation models to enhance learnability of the Convolutional Neural Network (CNN) and the Convolutional Long Short-Term Memory (ConvLSTM)-based deep learning models (DADLMs) and, by doing so, boost the accuracy of COVID-19 detection. Experimental results reveal improvement in terms of accuracy of detection, logarithmic loss, and testing time relative to DLMs devoid of such data augmentation. Furthermore, average increases of 4% to 11% in COVID-19 detection accuracy are reported in favour of the proposed data-augmented deep learning models relative to the machine learning techniques. Therefore, the proposed algorithm is effective in performing a rapid and consistent Corona virus diagnosis that is primarily aimed at assisting clinicians in making accurate identification of the virus.

Download Full-text

Application of Deep Learning for Characterization of Drivers’ Engagement in Secondary Tasks in In-Vehicle Systems

Transportation Research Record Journal of the Transportation Research Board ◽

10.1177/0361198120926507 ◽

2020 ◽

Vol 2674 (8) ◽

pp. 429-440

Author(s):

Osama A. Osman ◽

Hesham Rakha

Keyword(s):

Neural Network ◽

Deep Learning ◽

Time Series Data ◽

Short Term Memory ◽

Network Models ◽

Sensor Technology ◽

Series Data ◽

Neural Network Models ◽

Secondary Tasks ◽

Driving Assistance

Distracted driving (i.e., engaging in secondary tasks) is an epidemic that threatens the lives of thousands every year. Data collected from vehicular sensor technologies and through connectivity provide comprehensive information that, if used to detect driver engagement in secondary tasks, could save thousands of lives and millions of dollars. This study investigates the possibility of achieving this goal using promising deep learning tools. Specifically, two deep neural network models (a multilayer perceptron neural network model and a long short-term memory networks [LSTMN] model) were developed to identify three secondary tasks: cellphone calling, cellphone texting, and conversation with adjacent passengers. The Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS) time series data, collected using vehicle sensor technology, were used to train and test the model. The results show excellent performance for the developed models, with a slight improvement for the LSTMN model, with overall classification accuracies ranging between 95 and 96%. Specifically, the models are able to identify the different types of secondary tasks with high accuracies of 100% for calling, 96%–97% for texting, 90%–91% for conversation, and 95%–96% for the normal driving. Based on this performance, the developed models improve on the results of a previous model developed by the author to classify the same three secondary tasks, which had an accuracy of 82%. The model is promising for use in in-vehicle driving assistance technology to report engagement in unlawful tasks or alert drivers to take over control in level 1 and 2 automated vehicles.

Download Full-text

Sentiment Analysis of Lithuanian Texts Using Traditional and Deep Learning Approaches

Computers ◽

10.3390/computers8010004 ◽

2019 ◽

Vol 8 (1) ◽

pp. 4 ◽

Cited By ~ 4

Author(s):

Jurgita Kapočiūtė-Dzikienė ◽

Robertas Damaševičius ◽

Marcin Woźniak

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Sentiment Analysis ◽

Short Term Memory ◽

Machine Learning Techniques ◽

Support Vector ◽

Learning Approaches ◽

Full Dataset ◽

Learning Techniques ◽

Long Short Term Memory

We describe the sentiment analysis experiments that were performed on the Lithuanian Internet comment dataset using traditional machine learning (Naïve Bayes Multinomial—NBM and Support Vector Machine—SVM) and deep learning (Long Short-Term Memory—LSTM and Convolutional Neural Network—CNN) approaches. The traditional machine learning techniques were used with the features based on the lexical, morphological, and character information. The deep learning approaches were applied on the top of two types of word embeddings (Vord2Vec continuous bag-of-words with negative sampling and FastText). Both traditional and deep learning approaches had to solve the positive/negative/neutral sentiment classification task on the balanced and full dataset versions. The best deep learning results (reaching 0.706 of accuracy) were achieved on the full dataset with CNN applied on top of the FastText embeddings, replaced emoticons, and eliminated diacritics. The traditional machine learning approaches demonstrated the best performance (0.735 of accuracy) on the full dataset with the NBM method, replaced emoticons, restored diacritics, and lemma unigrams as features. Although traditional machine learning approaches were superior when compared to the deep learning methods; deep learning demonstrated good results when applied on the small datasets.

Download Full-text