Comparative Analysis of Recurrent Neural Network Architectures for Reservoir Inflow Forecasting

Due to the stochastic nature and complexity of flow, as well as the existence of hydrological uncertainties, predicting streamflow in dam reservoirs, especially in semi-arid and arid areas, is essential for the optimal and timely use of surface water resources. In this research, daily streamflow to the Ermenek hydroelectric dam reservoir located in Turkey is simulated using deep recurrent neural network (RNN) architectures, including bidirectional long short-term memory (Bi-LSTM), gated recurrent unit (GRU), long short-term memory (LSTM), and simple recurrent neural networks (simple RNN). For this purpose, daily observational flow data are used during the period 2012–2018, and all models are coded in Python software programming language. Only delays of streamflow time series are used as the input of models. Then, based on the correlation coefficient (CC), mean absolute error (MAE), root mean square error (RMSE), and Nash–Sutcliffe efficiency coefficient (NS), results of deep-learning architectures are compared with one another and with an artificial neural network (ANN) with two hidden layers. Results indicate that the accuracy of deep-learning RNN methods are better and more accurate than ANN. Among methods used in deep learning, the LSTM method has the best accuracy, namely, the simulated streamflow to the dam reservoir with 90% accuracy in the training stage and 87% accuracy in the testing stage. However, the accuracies of ANN in training and testing stages are 86% and 85%, respectively. Considering that the Ermenek Dam is used for hydroelectric purposes and energy production, modeling inflow in the most realistic way may lead to an increase in energy production and income by optimizing water management. Hence, multi-percentage improvements can be extremely useful. According to results, deep-learning methods of RNNs can be used for estimating streamflow to the Ermenek Dam reservoir due to their accuracy.

Download Full-text

A Deep Learning Approach for Intrusion Detection in Internet of Things using Bi-Directional Long Short-Term Memory Recurrent Neural Network

2018 28th International Telecommunication Networks and Applications Conference (ITNAC) ◽

10.1109/atnac.2018.8615294 ◽

2018 ◽

Cited By ~ 8

Author(s):

Bipraneel Roy ◽

Hon Cheung

Keyword(s):

Neural Network ◽

Deep Learning ◽

Internet Of Things ◽

Intrusion Detection ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Learning Approach ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Download Full-text

Combining Deep Learning Models for Enhancing the Detection of Botnet Attacks in Multiple Sensors Internet of Things Networks

JOIV International Journal on Informatics Visualization ◽

10.30630/joiv.5.4.733 ◽

2021 ◽

Vol 5 (4) ◽

pp. 380

Author(s):

Abdulkareem A. Hezam ◽

Salama A. Mostafa ◽

Zirawani Baharum ◽

Alde Alanda ◽

Mohd Zaki Salikon

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Denial Of Service ◽

Learning Models ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Iot Devices

Distributed-Denial-of-Service impacts are undeniably significant, and because of the development of IoT devices, they are expected to continue to rise in the future. Even though many solutions have been developed to identify and prevent this assault, which is mainly targeted at IoT devices, the danger continues to exist and is now larger than ever. It is common practice to launch denial of service attacks in order to prevent legitimate requests from being completed. This is accomplished by swamping the targeted machines or resources with false requests in an attempt to overpower systems and prevent many or all legitimate requests from being completed. There have been many efforts to use machine learning to tackle puzzle-like middle-box problems and other Artificial Intelligence (AI) problems in the last few years. The modern botnets are so sophisticated that they may evolve daily, as in the case of the Mirai botnet, for example. This research presents a deep learning method based on a real-world dataset gathered by infecting nine Internet of Things devices with two of the most destructive DDoS botnets, Mirai and Bashlite, and then analyzing the results. This paper proposes the BiLSTM-CNN model that combines Bidirectional Long-Short Term Memory Recurrent Neural Network and Convolutional Neural Network (CNN). This model employs CNN for data processing and feature optimization, and the BiLSTM is used for classification. This model is evaluated by comparing its results with three standard deep learning models of CNN, Recurrent Neural Network (RNN), and long-Short Term Memory Recurrent Neural Network (LSTM–RNN). There is a huge need for more realistic datasets to fully test such models' capabilities, and where N-BaIoT comes, it also includes multi-device IoT data. The N-BaIoT dataset contains DDoS attacks with the two of the most used types of botnets: Bashlite and Mirai. The 10-fold cross-validation technique tests the four models. The obtained results show that the BiLSTM-CNN outperforms all other individual classifiers in every aspect in which it achieves an accuracy of 89.79% and an error rate of 0.1546 with a very high precision of 93.92% with an f1-score and recall of 85.73% and 89.11%, respectively. The RNN achieves the highest accuracy among the three individual models, with an accuracy of 89.77%, followed by LSTM, which achieves the second-highest accuracy of 89.71%. CNN, on the other hand, achieves the lowest accuracy among all classifiers of 89.50%.

Download Full-text

Deep Learning for Face Expressions Detection: Enhanced Recurrent Neural Network with Long Short Term Memory

Communications in Computer and Information Science - Applied Computing to Support Industry: Innovation and Technology ◽

10.1007/978-3-030-38752-5_19 ◽

2020 ◽

pp. 237-247

Author(s):

Wafaa Mahdi Salih ◽

Ibraheem Nadher ◽

Ahmed Tariq

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

Face Expressions

Download Full-text

Hand Gesture Recognition in Video Sequences Using Deep Convolutional and Recurrent Neural Networks

Applied Computer Systems ◽

10.2478/acss-2020-0007 ◽

2020 ◽

Vol 25 (1) ◽

pp. 57-61

Author(s):

Falah Obaid ◽

Amin Babadi ◽

Ahmad Yoosofan

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Gesture Recognition ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Hand Gesture ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

AbstractDeep learning is a new branch of machine learning, which is widely used by researchers in a lot of artificial intelligence applications, including signal processing and computer vision. The present research investigates the use of deep learning to solve the hand gesture recognition (HGR) problem and proposes two models using deep learning architecture. The first model comprises a convolutional neural network (CNN) and a recurrent neural network with a long short-term memory (RNN-LSTM). The accuracy of model achieves up to 82 % when fed by colour channel, and 89 % when fed by depth channel. The second model comprises two parallel convolutional neural networks, which are merged by a merge layer, and a recurrent neural network with a long short-term memory fed by RGB-D. The accuracy of the latest model achieves up to 93 %.

Download Full-text

Implementasi Metode Reccurrent Neural Network pada Text Summarization dengan Teknik Abstraktif

Jurnal Teknologi Informasi dan Ilmu Komputer ◽

10.25126/jtiik.2019641067 ◽

2019 ◽

Vol 6 (4) ◽

pp. 377

Author(s):

Kasyfi Ivanedra ◽

Metty Mustikasari

Keyword(s):

Neural Network ◽

Artificial Intelligence ◽

Deep Learning ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Text Summarization ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory ◽

F Measure

Text Summarization atau peringkas text merupakan salah satu penerapan Artificial Intelligence (AI) dimana komputer dapat meringkas text pada suatu kalimat atau artikel menjadi lebih sederhana dengan tujuan untuk mempermudah manusia dalam mengambil kesimpulan dari artikel yang panjang tanpa harus membaca secara keseluruhan. Peringkasan teks secara otomatis dengan menggunakan teknik Abstraktif memiliki kemampuan meringkas teks lebih natural sebagaimana manusia meringkas dibandingkan dengan teknik ekstraktif yang hanya menyusun kalimat berdasarkan frekuensi kemunculan kata. Untuk dapat menghasilkan sistem peringkas teks dengan metode abstraktif, membutuhkan metode Recurrent Neural Network (RNN) yang memiliki sistematika perhitungan bobot secara berulang. RNN merupakan bagian dari Deep Learning dimana nilai akurasi yang dihasilkan dapat lebih baik dibandingkan dengan jaringan saraf tiruan sederhana karena bobot yang dihitung akan lebih akurat mendekati persamaan setiap kata. Jenis RNN yang digunakan adalah LSTM (Long Short Term Memory) untuk menutupi kekurangan pada RNN yang tidak dapat menyimpan memori untuk dipilah dan menambahkan mekanisme Attention agar setiap kata dapat lebih fokus pada konteks. Penelitian ini menguji performa sistem menggunakan Precision, Recall, dan F-Measure dengan membandingan hasil ringkasan yang dihasilkan oleh sistem dan ringkasan yang dibuat oleh manusia. Dataset yang digunakan adalah data artikel berita dengan jumlah total artikel sebanyak 4515 buah artikel. Pengujian dibagi berdasarkan data dengan menggunakan Stemming dan dengan teknik Non-stemming. Nilai rata-rata recall artikel berita non-stemming adalah sebesar 41%, precision sebesar 81%, dan F-measure sebesar 54,27%. Sedangkan nilai rata-rata recall artikel berita dengan teknik stemming sebesar 44%, precision sebesar 88%, dan F-measure sebesar 58,20 %.AbstractText Summarization is the application of Artificial Intelligence (AI) where the computer can summarize text of article to make it easier for humans to draw conclusions from long articles without having to read entirely. Abstractive techniques has ability to summarize the text more naturally as humans summarize. The summary results from abstractive techinques are more in context when compared to extractive techniques which only arrange sentences based on the frequency of occurrence of the word. To be able to produce a text summarization system with an abstractive techniques, it is required Deep Learning by using the Recurrent Neural Network (RNN) rather than simple Artificial Neural Network (ANN) method which has a systematic calculation of weight repeatedly in order to improve accuracy. The type of RNN used is LSTM (Long Short Term Memory) to cover the shortcomings of the RNN which cannot store memory to be sorted and add an Attention mechanism so that each word can focus more on the context.This study examines the performance of Precision, Recall, and F-Measure from the comparison of the summary results produced by the system and summaries made by humans. The dataset used is news article data with 4515 articles. Testing was divided based on data using Stemming and Non-stemming techniques. The average recall value of non-stemming news articles is 41%, precision is 81%, and F-measure is 54.27%. While the average value of recall of news articles with stemming technique is 44%, precision is 88%, and F-measure is 58.20%.

Download Full-text

Deep Learning with Long Short-Term Memory Recurrent Neural Network for Daily Container Volumes of Storage Yard Predictions in Port

2018 International Conference on Cyberworlds (CW) ◽

10.1109/cw.2018.00083 ◽

2018 ◽

Author(s):

Yinping Gao ◽

Daofang Chang ◽

Chun-Hsien Chen ◽

Ting Fang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recurrent Neural Network ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Download Full-text

Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network

Remote Sensing ◽

10.3390/rs13101953 ◽

2021 ◽

Vol 13 (10) ◽

pp. 1953

Author(s):

Seyed Majid Azimi ◽

Maximilian Kraus ◽

Reza Bahmanyar ◽

Peter Reinartz

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Object Tracking ◽

Short Term Memory ◽

Aerial Imagery ◽

Future Research ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method AerialMPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of AerialMPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the L1 and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Tracking datasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, L1 yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research.

Download Full-text