scholarly journals A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling

Forecasting ◽  
2021 ◽  
Vol 4 (1) ◽  
pp. 1-25
Author(s):  
Thabang Mathonsi ◽  
Terence L. van Zyl
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Hybrid Methods ◽  
Prediction Interval ◽  
Hybrid Approach ◽  
Computational Cost ◽  
Exponential Smoothing ◽  
Learning Methods

Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include (i) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, (ii) challenges associated with auto-correlation inherent in the data, as well as (iii) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.

Improving Sentiment Analysis using Hybrid Deep Learning Model

2020 ◽  
Vol 13 (4) ◽  
pp. 627-640 ◽  
Author(s):  
Avinash Chandra Pandey ◽  
Dharmveer Singh Rajpoot
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Classification Accuracy ◽  
Short Term Memory ◽  
Computational Cost ◽  
Extraction Process ◽  
Learning Model ◽  
Sentiment Classification ◽  
Deep Learning Model

Background: Sentiment analysis is a contextual mining of text which determines viewpoint of users with respect to some sentimental topics commonly present at social networking websites. Twitter is one of the social sites where people express their opinion about any topic in the form of tweets. These tweets can be examined using various sentiment classification methods to find the opinion of users. Traditional sentiment analysis methods use manually extracted features for opinion classification. The manual feature extraction process is a complicated task since it requires predefined sentiment lexicons. On the other hand, deep learning methods automatically extract relevant features from data hence; they provide better performance and richer representation competency than the traditional methods. Objective: The main aim of this paper is to enhance the sentiment classification accuracy and to reduce the computational cost. Method: To achieve the objective, a hybrid deep learning model, based on convolution neural network and bi-directional long-short term memory neural network has been introduced. Results: The proposed sentiment classification method achieves the highest accuracy for the most of the datasets. Further, from the statistical analysis efficacy of the proposed method has been validated. Conclusion: Sentiment classification accuracy can be improved by creating veracious hybrid models. Moreover, performance can also be enhanced by tuning the hyper parameters of deep leaning models.

scholarly journals A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Electronics ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 81
Author(s):  
Jianbin Xiong ◽  
Dezheng Yu ◽  
Shuangyin Liu ◽  
Lei Shu ◽  
Xiaochan Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Plant Species ◽  
Image Recognition ◽  
Recurrent Neural Network ◽  
Plant Diseases ◽  
Learning Methods ◽  
Smart Agriculture ◽  
Important Branch

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

scholarly journals Comparative Analysis of Recurrent Neural Network Architectures for Reservoir Inflow Forecasting

Water ◽  
2020 ◽  
Vol 12 (5) ◽  
pp. 1500 ◽  
Author(s):  
Halit Apaydin ◽  
Hajar Feizi ◽  
Mohammad Taghi Sattari ◽  
Muslume Sevba Colak ◽  
Shahaboddin Shamshirband ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Energy Production ◽  
Short Term Memory ◽  
Efficiency Coefficient ◽  
Dam Reservoir ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Due to the stochastic nature and complexity of flow, as well as the existence of hydrological uncertainties, predicting streamflow in dam reservoirs, especially in semi-arid and arid areas, is essential for the optimal and timely use of surface water resources. In this research, daily streamflow to the Ermenek hydroelectric dam reservoir located in Turkey is simulated using deep recurrent neural network (RNN) architectures, including bidirectional long short-term memory (Bi-LSTM), gated recurrent unit (GRU), long short-term memory (LSTM), and simple recurrent neural networks (simple RNN). For this purpose, daily observational flow data are used during the period 2012–2018, and all models are coded in Python software programming language. Only delays of streamflow time series are used as the input of models. Then, based on the correlation coefficient (CC), mean absolute error (MAE), root mean square error (RMSE), and Nash–Sutcliffe efficiency coefficient (NS), results of deep-learning architectures are compared with one another and with an artificial neural network (ANN) with two hidden layers. Results indicate that the accuracy of deep-learning RNN methods are better and more accurate than ANN. Among methods used in deep learning, the LSTM method has the best accuracy, namely, the simulated streamflow to the dam reservoir with 90% accuracy in the training stage and 87% accuracy in the testing stage. However, the accuracies of ANN in training and testing stages are 86% and 85%, respectively. Considering that the Ermenek Dam is used for hydroelectric purposes and energy production, modeling inflow in the most realistic way may lead to an increase in energy production and income by optimizing water management. Hence, multi-percentage improvements can be extremely useful. According to results, deep-learning methods of RNNs can be used for estimating streamflow to the Ermenek Dam reservoir due to their accuracy.

scholarly journals A Comparative Study of Bitcoin Price Prediction Using Deep Learning

Mathematics ◽  
2019 ◽  
Vol 7 (10) ◽  
pp. 898 ◽  
Author(s):  
Suhwan Ji ◽  
Jongmin Kim ◽  
Hyeonseung Im
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Prediction Models ◽  
Short Term Memory ◽  
Learning Methods ◽  
The Public ◽  
Price Prediction ◽  
Profitability Analysis ◽  
The Media ◽  
Long Short Term Memory

Bitcoin has recently received a lot of attention from the media and the public due to its recent price surge and crash. Correspondingly, many researchers have investigated various factors that affect the Bitcoin price and the patterns behind its fluctuations, in particular, using various machine learning methods. In this paper, we study and compare various state-of-the-art deep learning methods such as a deep neural network (DNN), a long short-term memory (LSTM) model, a convolutional neural network, a deep residual network, and their combinations for Bitcoin price prediction. Experimental results showed that although LSTM-based prediction models slightly outperformed the other prediction models for Bitcoin price prediction (regression), DNN-based models performed the best for price ups and downs prediction (classification). In addition, a simple profitability analysis showed that classification models were more effective than regression models for algorithmic trading. Overall, the performances of the proposed deep learning-based prediction models were comparable.

scholarly journals Hybrid deep learning model using recurrent neural network and gated recurrent unit for heart disease prediction

2021 ◽  
Vol 11 (6) ◽  
pp. 5467
Author(s):  
Surenthiran Krishnan ◽  
Pritheega Magalingam ◽  
Roslina Ibrahim
Keyword(s):  
Neural Network ◽  
Heart Disease ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Learning Model ◽  
Disease Prediction ◽  
The Neural Network ◽  
Proposed Model ◽  
Deep Learning Model

<span>This paper proposes a new hybrid deep learning model for heart disease prediction using recurrent neural network (RNN) with the combination of multiple gated recurrent units (GRU), long short-term memory (LSTM) and Adam optimizer. This proposed model resulted in an outstanding accuracy of 98.6876% which is the highest in the existing model of RNN. The model was developed in Python 3.7 by integrating RNN in multiple GRU that operates in Keras and Tensorflow as the backend for deep learning process, supported by various Python libraries. The recent existing models using RNN have reached an accuracy of 98.23% and deep neural network (DNN) has reached 98.5%. The common drawbacks of the existing models are low accuracy due to the complex build-up of the neural network, high number of neurons with redundancy in the neural network model and imbalance datasets of Cleveland. Experiments were conducted with various customized model, where results showed that the proposed model using RNN and multiple GRU with synthetic minority oversampling technique (SMOTe) has reached the best performance level. This is the highest accuracy result for RNN using Cleveland datasets and much promising for making an early heart disease prediction for the patients.</span>

scholarly journals Application of deep learning methods to predict ionosphere parameters in real time

2020 ◽  
Vol 196 ◽  
pp. 02007
Author(s):  
Vladimir Mochalov ◽  
Anastasia Mochalova
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Real Time ◽  
Network Architecture ◽  
Short Term Memory ◽  
Neural Network Architecture ◽  
Short Term ◽  
Learning Methods ◽  
Term Memory ◽  
Long Short Term Memory

In this paper, the previously obtained results on recognition of ionograms using deep learning are expanded to predict the parameters of the ionosphere. After the ionospheric parameters have been identified on the ionogram using deep learning in real time, we can predict the parameters for some time ahead on the basis of the new data obtained Examples of predicting the ionosphere parameters using an artificial recurrent neural network architecture long short-term memory are given. The place of the block for predicting the parameters of the ionosphere in the system for analyzing ionospheric data using deep learning methods is shown.

scholarly journals Research on the Estimate of Gas Hydrate Saturation Based on LSTM Recurrent Neural Network

Energies ◽  
2020 ◽  
Vol 13 (24) ◽  
pp. 6536
Author(s):  
Chuanhui Li ◽  
Xuewei Liu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Gas Hydrate ◽  
Short Term Memory ◽  
Chloride Concentration ◽  
Well Logs ◽  
Good Prediction ◽  
Learning Technology ◽  
Hydrate Saturation

Gas hydrate saturation is an important index for evaluating gas hydrate reservoirs, and well logs are an effective method for estimating gas hydrate saturation. To use well logs better to estimate gas hydrate saturation, and to establish the deep internal connections and laws of the data, we propose a method of using deep learning technology to estimate gas hydrate saturation from well logs. Considering that well logs have sequential characteristics, we used the long short-term memory (LSTM) recurrent neural network to predict the gas hydrate saturation from the well logs of two sites in the Shenhu area, South China Sea. By constructing an LSTM recurrent layer and two fully connected layers at one site, we used resistivity and acoustic velocity logs that were sensitive to gas hydrate as input. We used the gas hydrate saturation calculated by the chloride concentration of the pore water as output to train the LSTM network. We achieved a good training result. Applying the trained LSTM recurrent neural network to another site in the same area achieved good prediction of gas hydrate saturation, showing the unique advantages of deep learning technology in gas hydrate saturation estimation.

Combining Deep Learning Models for Enhancing the Detection of Botnet Attacks in Multiple Sensors Internet of Things Networks

2021 ◽  
Vol 5 (4) ◽  
pp. 380
Author(s):  
Abdulkareem A. Hezam ◽  
Salama A. Mostafa ◽  
Zirawani Baharum ◽  
Alde Alanda ◽  
Mohd Zaki Salikon
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Denial Of Service ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Iot Devices

Distributed-Denial-of-Service impacts are undeniably significant, and because of the development of IoT devices, they are expected to continue to rise in the future. Even though many solutions have been developed to identify and prevent this assault, which is mainly targeted at IoT devices, the danger continues to exist and is now larger than ever. It is common practice to launch denial of service attacks in order to prevent legitimate requests from being completed. This is accomplished by swamping the targeted machines or resources with false requests in an attempt to overpower systems and prevent many or all legitimate requests from being completed. There have been many efforts to use machine learning to tackle puzzle-like middle-box problems and other Artificial Intelligence (AI) problems in the last few years. The modern botnets are so sophisticated that they may evolve daily, as in the case of the Mirai botnet, for example. This research presents a deep learning method based on a real-world dataset gathered by infecting nine Internet of Things devices with two of the most destructive DDoS botnets, Mirai and Bashlite, and then analyzing the results. This paper proposes the BiLSTM-CNN model that combines Bidirectional Long-Short Term Memory Recurrent Neural Network and Convolutional Neural Network (CNN). This model employs CNN for data processing and feature optimization, and the BiLSTM is used for classification. This model is evaluated by comparing its results with three standard deep learning models of CNN, Recurrent Neural Network (RNN), and long-Short Term Memory Recurrent Neural Network (LSTM–RNN). There is a huge need for more realistic datasets to fully test such models' capabilities, and where N-BaIoT comes, it also includes multi-device IoT data. The N-BaIoT dataset contains DDoS attacks with the two of the most used types of botnets: Bashlite and Mirai. The 10-fold cross-validation technique tests the four models. The obtained results show that the BiLSTM-CNN outperforms all other individual classifiers in every aspect in which it achieves an accuracy of 89.79% and an error rate of 0.1546 with a very high precision of 93.92% with an f1-score and recall of 85.73% and 89.11%, respectively. The RNN achieves the highest accuracy among the three individual models, with an accuracy of 89.77%, followed by LSTM, which achieves the second-highest accuracy of 89.71%. CNN, on the other hand, achieves the lowest accuracy among all classifiers of 89.50%.

scholarly journals A Spectral-Spatial Cascaded 3D Convolutional Neural Network with a Convolutional Long Short-Term Memory Network for Hyperspectral Image Classification

2019 ◽  
Vol 11 (20) ◽  
pp. 2363 ◽  
Author(s):  
Wenchao Qi ◽  
Xia Zhang ◽  
Nan Wang ◽  
Mao Zhang ◽  
Yi Cen
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Short Term Memory ◽  
Hyperspectral Image ◽  
Short Term ◽  
Dynamic Learning ◽  
Learning Methods ◽  
Term Memory ◽  
Long Short Term Memory

Deep learning methods used for hyperspectral image (HSI) classification often achieve greater accuracy than traditional algorithms but require large numbers of training epochs. To simplify model structures and reduce their training epochs, an end-to-end deep learning framework incorporating a spectral-spatial cascaded 3D convolutional neural network (CNN) with a convolutional long short-term memory (CLSTM) network, called SSCC, is proposed herein for HSI classification. The SSCC framework employs cascaded 3D CNN to learn the spectral-spatial features of HSIs and uses the CLSTM network to extract sequence features. Residual connections are used in SSCC to accelerate model convergence, with the outputs of previous convolutional layers concatenated as inputs for subsequent layers. Moreover, the data augmentation, parametric rectified linear unit, dynamic learning rate, batch normalization, and regularization (including dropout and L2) methods are used to increase classification accuracy and prevent overfitting. These attributes allow the SSCC framework to achieve good performance for HSI classification within 20 epochs. Three well-known datasets including Indiana Pines, University of Pavia, and Pavia Center were employed to evaluate the classification performance of the proposed algorithm. The GF-5 dataset of Anxin County, obtained from China’s recently launched spaceborne Advanced Hyperspectral Imager, was also used for classification experiments. The experimental results demonstrate that the proposed SSCC framework achieves state-of-the-art performance with better training efficiency than other deep learning methods.

Sign in / Sign up

Export Citation Format

Share Document