Hybrid Ensemble Deep Learning-Based Approach for Time Series Energy Prediction

The energy manufacturers are required to produce an accurate amount of energy by meeting the energy requirements at the end-user side. Consequently, energy prediction becomes an essential role in the electric industrial zone. In this paper, we propose the hybrid ensemble deep learning model, which combines multilayer perceptron (MLP), convolutional neural network (CNN), long short-term memory (LSTM), and hybrid CNN-LSTM to improve the forecasting performance. These DL architectures are more popular and better than other machine learning (ML) models for time series electrical load prediction. Therefore, hourly-based energy data are collected from Jeju Island, South Korea, and applied for forecasting. We considered external features associated with meteorological conditions affecting energy. Two-year training and one-year testing data are preprocessed and arranged to reform the times series, which are then trained in each DL model. The forecasting results of the proposed ensemble model are evaluated by using mean square error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). Error metrics are compared with DL stand-alone models such as MLP, CNN, LSTM, and CNN-LSTM. Our ensemble model provides better performance than other forecasting models, providing minimum MAPE at 0.75%, and was proven to be inherently symmetric for forecasting time-series energy and demand data, which is of utmost concern to the power system sector.

Download Full-text

Wind Power Short-Term Prediction Based on LSTM and Discrete Wavelet Transform

Applied Sciences ◽

10.3390/app9061108 ◽

2019 ◽

Vol 9 (6) ◽

pp. 1108 ◽

Cited By ~ 30

Author(s):

Yao Liu ◽

Lin Guan ◽

Chen Hou ◽

Hua Han ◽

Zhangjie Liu ◽

...

Keyword(s):

Time Series ◽

Wavelet Transform ◽

Discrete Wavelet Transform ◽

Wind Power ◽

Short Term Memory ◽

Wind Farms ◽

Absolute Error ◽

Percentage Error ◽

Discrete Wavelet ◽

Short Term

A wind power short-term forecasting method based on discrete wavelet transform and long short-term memory networks (DWT_LSTM) is proposed. The LSTM network is designed to effectively exhibit the dynamic behavior of the wind power time series. The discrete wavelet transform is introduced to decompose the non-stationary wind power time series into several components which have more stationarity and are easier to predict. Each component is dug by an independent LSTM. The forecasting results of the wind power are obtained by synthesizing the prediction values of all components. The prediction accuracy has been improved by the proposed method, which is validated by the MAE (mean absolute error), MAPE (mean absolute percentage error), and RMSE (root mean square error) of experimental results of three wind farms as the benchmarks. Wind power forecasting based on the proposed method provides an alternative way to improve the security and stability of the electric power network with the high penetration of wind power.

Download Full-text

Examination and Forecasting of Drug Consumption Based on Recurrent Deep Learning

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.b1069.0982s1019 ◽

2019 ◽

Vol 8 (2S10) ◽

pp. 414-420

Keyword(s):

Time Series ◽

Deep Learning ◽

Time Series Analysis ◽

Evaluation Method ◽

Short Term Memory ◽

Prediction Method ◽

Pharmaceutical Drugs ◽

Data Set ◽

Series Analysis ◽

One Year

The provision of pharmaceutical drugs in quantities appropriate to consumption is an important point in the pharmaceutical industry and storage of medicines, as the production of large quantities of unnecessary drugs leads to a longer storage of drugs. Meanwhile most medicines have a short shelf life. When the amount of production is less than required, this affects the satisfaction of the customer and the marketing of the drug. Time series analysis is the appropriate solution to this problem. Deep learning has been adapted for the purpose of time series analysis and a prediction of the required quantities drugs. A recurrent neural network with Long-Short Term Memory LSTM has been used by deep learning. The proposed methodology is based on the seasonal number of prescription required quantities with the number of quarters as indicators. The aim of the research is to forecast the drugs amount needed for one year. The proposed method is assessed using two types of evaluation. The first one is based on MSE and the visualization of the actual data and forecasted data. The proposed method has reached a low value of MSE and the visualization graph is semi-identical, whereas the second evaluation method compares the result of the proposed method with traditional forecasting method. Multiple linear regression is a traditional prediction method used with the data set, whose results are relatively good and promising compared to the results of the traditional method.

Download Full-text

Forecasting Gold Prices in India using Time series and Deep Learning Algorithms

International Journal of Engineering and Advanced Technology - Regular Issue ◽

10.35940/ijeat.d2537.0610521 ◽

2021 ◽

Vol 10 (5) ◽

pp. 21-27

Author(s):

P Sai Shankar ◽

M Krishna Reddy

Keyword(s):

Time Series ◽

Deep Learning ◽

Learning Algorithm ◽

Arima Model ◽

Absolute Error ◽

Percentage Error ◽

Deep Learning Algorithm ◽

Primary Object ◽

Daily Data ◽

Forecasting Performance

The primary object of this paper is to compare the traditional time series models with deep learning algorithm. The ARIMA model is developed to forecast Indian Gold prices using daily data for the period 2016 to 2020 obtained from World Gold Council. We fitted the ARIMA (2,1,2) model which exhibited the least AIC values. In the meanwhile, MLP, CNN and LSTM models are also examined to forecast the gold prices in India. Mean absolute error, mean absolute percentage error and root mean squared errors used to evaluate the forecasting performance of the models. Hence, LSTM model superior than that of the other three models for forecasting the gold prices in India.

Download Full-text

The Deep Learning LSTM and MTD Models Best Predict Acute Respiratory Infection among Under-Five-Year Old Children in Somaliland

Symmetry ◽

10.3390/sym13071156 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1156

Author(s):

Mohamed Yusuf Hassan

Keyword(s):

Time Series ◽

Deep Learning ◽

Respiratory Infection ◽

Acute Respiratory Infection ◽

Short Term Memory ◽

Mean Deviation ◽

Sarima Model ◽

Under Five ◽

Competing Models ◽

Long Short Term Memory

The most effective techniques for predicting time series patterns include machine learning and classical time series methods. The aim of this study is to search for the best artificial intelligence and classical forecasting techniques that can predict the spread of acute respiratory infection (ARI) and pneumonia among under-five-year old children in Somaliland. The techniques used in the study include seasonal autoregressive integrated moving averages (SARIMA), mixture transitions distribution (MTD), and long short term memory (LSTM) deep learning. The data used in the study were monthly observations collected from five regions in Somaliland from 2011–2014. Prediction results from the three best competing models are compared by using root mean square error (RMSE) and absolute mean deviation (MAD) accuracy measures. Results have shown that the deep learning LSTM and MTD models slightly outperformed the classical SARIMA model in predicting ARI values.

Download Full-text

Prognostics and RUL Estimations of SAC305, SAC105 and SnAg Solders Under Temperature and Vibration Using Long Short-Term Memory (LSTM) Deep Learning

10.1115/ipack2021-74066 ◽

2021 ◽

Author(s):

Pradeep Lall ◽

Tony Thomas ◽

Ken Blecker

Keyword(s):

Time Series ◽

Deep Learning ◽

Short Term Memory ◽

Remaining Useful Life ◽

Operating Conditions ◽

Short Term ◽

Term Memory ◽

Rectangular Pattern ◽

Useful Life ◽

Long Short Term Memory

Abstract Prognostics and Remaining Useful Life (RUL) estimations of complex systems are essential to operational safety, increased efficiency, and help to schedule maintenance proactively. Modeling the remaining useful life of a system with many complexities is possible with the rapid development in the field of deep learning as a computational technique for failure prediction. Deep learning can adapt to multivariate parameters complex and nonlinear behavior, which is difficult using traditional time-series models for forecasting and prediction purposes. In this paper, a deep learning approach based on Long Short-Term Memory (LSTM) network is used to predict the remaining useful life of the PCB at different conditions of temperature and vibration. This technique can identify the different underlying patterns in the time series that can predict the RUL. This study involves feature vector identification and RUL estimations for SAC305, SAC105, and Tin Lead solder PCBs under different vibration levels and temperature conditions. The acceleration levels of vibration are fixed at 5g and 10g, while the temperature levels are 55°C and 100°C. The test board is a multilayer FR4 configuration with JEDEC standard dimensions consists of twelve packages arranged in a rectangular pattern. Strain signals are acquired from the backside of the PCB at symmetric locations to identify the failure of all the packages during vibration. The strain signals are resistance values that are acquired simultaneously during the experiment until the failure of most of the packages on the board. The feature vectors are identified from statistical analysis on the strain signals frequency and instantaneous frequency components. The principal component analysis is used as a data reduction technique to identify the different patterns produced from the four strain signals with failures of the packages during vibration. LSTM deep learning method is used to model the RUL of the packages at different individual operating conditions of vibration for all three solder materials involved in this study. A combined model for RUL prediction for a material that can take care of the changes in the operating conditions is also modeled for each material.

Download Full-text

Efficient Deep Learning Models for DGA Domain Detection

Security and Communication Networks ◽

10.1155/2021/8887881 ◽

2021 ◽

Vol 2021 ◽

pp. 1-15

Author(s):

Juhong Namgung ◽

Siwoon Son ◽

Yang-Sae Moon

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Ensemble Model ◽

Learning Models ◽

Short Term ◽

Domain Names ◽

Additional Information ◽

Domain Sequence ◽

Long Short Term Memory ◽

And Control

In recent years, cyberattacks using command and control (C&C) servers have significantly increased. To hide their C&C servers, attackers often use a domain generation algorithm (DGA), which automatically generates domain names for the C&C servers. Accordingly, extensive research on DGA domain detection has been conducted. However, existing methods cannot accurately detect continuously generated DGA domains and can easily be evaded by an attacker. Recently, long short-term memory- (LSTM-) based deep learning models have been introduced to detect DGA domains in real time using only domain names without feature extraction or additional information. In this paper, we propose an efficient DGA domain detection method based on bidirectional LSTM (BiLSTM), which learns bidirectional information as opposed to unidirectional information learned by LSTM. We further maximize the detection performance with a convolutional neural network (CNN) + BiLSTM ensemble model using Attention mechanism, which allows the model to learn both local and global information in a domain sequence. Experimental results show that existing CNN and LSTM models achieved F1-scores of 0.9384 and 0.9597, respectively, while the proposed BiLSTM and ensemble models achieved higher F1-scores of 0.9618 and 0.9666, respectively. In addition, the ensemble model achieved the best performance for most DGA domain classes, enabling more accurate DGA domain detection than existing models.

Download Full-text

PP15 Predicting variations of calls to an ambulance service in the UK caused by circulating infections using-deep learning methods

Emergency Medicine Journal ◽

10.1136/emermed-2019-999abs.15 ◽

2019 ◽

Vol 36 (10) ◽

pp. e7.2-e7

Author(s):

Thilo Reich ◽

Marcin Budka

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Prehospital Care ◽

Machine Learning Algorithms ◽

Ambulance Service ◽

Percentage Error ◽

Learning Methods ◽

Average Percentage ◽

Average Percentage Error ◽

The Uk

BackgroundDigital patient records in the ambulance service have opened up new opportunities for prehospital care. Previously it was demonstrated that prehospital pyrexia numbers are linked to an increase in overall calls to the ambulance service. This study aims to predict the future number of calls using deep-learning methods.MethodsTemperature readings for 280,447 patients were generously provided by the South Western Ambulance Service Trust. The data covered the time between 05/01/2016 and 30/04/2017 with overall 44,472 patients being pyretic. A rolling window of 10 days was applied to daily sums for both pyretic and apyretic patients. These windows were used as input features to train machine-learning algorithms predicting the number of calls 10 days ahead. Algorithms tested include Linear Regression (LR), basic Recurrent Neural Networks (RNN), Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures. A genetic approach was used to optimise the architecture, in which parameters were randomly modified and over several generations the best performing algorithm will be selected to be further manipulated. To assess performance the Mean Average Percentage Error (MAPE) was used.ResultsThe initial analysis showed that the total patient number and pyretic patient numbers are correlated. The best performing algorithms with varying numbers of hidden units had the following MAPE in comparison to simple LR: LR=19.4%, LSTM (104 units) = 6.1%, RNN (79 units)=6.01%, GRU (80 units)=5.97%.ConclusionsThese preliminary results suggest that deep-learning methods allow to predict the variations in total number of calls caused by circulating infections. Further investigations will aim to confirm these findings. Once fully verified these algorithms could play a major role in operational planning of any ambulance service by predicting increases in demand.

Download Full-text

Ranking Information Extracted from Uncertainty Quantification of the Prediction of a Deep Learning Model on Medical Time Series Data

Mathematics ◽

10.3390/math8071078 ◽

2020 ◽

Vol 8 (7) ◽

pp. 1078

Author(s):

Ruxandra Stoean ◽

Catalin Stoean ◽

Miguel Atencia ◽

Roberto Rodríguez-Labrada ◽

Gonzalo Joya

Keyword(s):

Time Series ◽

Monte Carlo ◽

Deep Learning ◽

Uncertainty Quantification ◽

Time Series Data ◽

Short Term Memory ◽

Learning Model ◽

Series Data ◽

Real World Problem ◽

Uncertainty Estimates

Uncertainty quantification in deep learning models is especially important for the medical applications of this complex and successful type of neural architectures. One popular technique is Monte Carlo dropout that gives a sample output for a record, which can be measured statistically in terms of average probability and variance for each diagnostic class of the problem. The current paper puts forward a convolutional–long short-term memory network model with a Monte Carlo dropout layer for obtaining information regarding the model uncertainty for saccadic records of all patients. These are next used in assessing the uncertainty of the learning model at the higher level of sets of multiple records (i.e., registers) that are gathered for one patient case by the examining physician towards an accurate diagnosis. Means and standard deviations are additionally calculated for the Monte Carlo uncertainty estimates of groups of predictions. These serve as a new collection where a random forest model can perform both classification and ranking of variable importance. The approach is validated on a real-world problem of classifying electrooculography time series for an early detection of spinocerebellar ataxia 2 and reaches an accuracy of 88.59% in distinguishing between the three classes of patients.

Download Full-text

Ultra-Short-Term Load Demand Forecast Model Framework Based on Deep Learning

Energies ◽

10.3390/en13184900 ◽

2020 ◽

Vol 13 (18) ◽

pp. 4900 ◽

Cited By ~ 1

Author(s):

Hongze Li ◽

Hongyu Liu ◽

Hongyan Ji ◽

Shiying Zhang ◽

Pengfei Li

Keyword(s):

Time Series ◽

Deep Learning ◽

Meteorological Data ◽

Demand Forecasting ◽

Dense Layer ◽

Load Prediction ◽

Demand Forecast ◽

Short Term ◽

Model Framework ◽

Load Demand

Ultra-short-term load demand forecasting is significant to the rapid response and real-time dispatching of the power demand side. Considering too many random factors that affect the load, this paper combines convolution, long short-term memory (LSTM), and gated recurrent unit (GRU) algorithms to propose an ultra-short-term load forecasting model based on deep learning. Firstly, more than 100,000 pieces of historical load and meteorological data from Beijing in the three years from 2016 to 2018 were collected, and the meteorological data were divided into 18 types considering the actual meteorological characteristics of Beijing. Secondly, after the standardized processing of the time-series samples, the convolution filter was used to extract the features of the high-order samples to reduce the number of training parameters. On this basis, the LSTM layer and GRU layer were used for modeling based on time series. A dropout layer was introduced after each layer to reduce the risk of overfitting. Finally, load prediction results were output as a dense layer. In the model training process, the mean square error (MSE) was used as the objective optimization function to train the deep learning model and find the optimal super parameter. In addition, based on the average training time, training error, and prediction error, this paper verifies the effectiveness and practicability of the load prediction model proposed under the deep learning structure in this paper by comparing it with four other models including GRU, LSTM, Conv-GRU, and Conv-LSTM.

Download Full-text

Deep Learning-Based Maximum Temperature Forecasting Assisted with Meta-Learning for Hyperparameter Optimization

Atmosphere ◽

10.3390/atmos11050487 ◽

2020 ◽

Vol 11 (5) ◽

pp. 487 ◽

Cited By ~ 3

Author(s):

Trang Thi Kieu Tran ◽

Taesam Lee ◽

Ju-Young Shin ◽

Jong-Suk Kim ◽

Mohamad Kamruzzaman

Keyword(s):

Neural Network ◽

Time Series ◽

Deep Learning ◽

Short Term Memory ◽

Human Life ◽

Time Series Prediction ◽

Vital Role ◽

Maximum Temperature ◽

Hyperparameter Optimization ◽

Meta Learning

Time series forecasting of meteorological variables such as daily temperature has recently drawn considerable attention from researchers to address the limitations of traditional forecasting models. However, a middle-range (e.g., 5–20 days) forecasting is an extremely challenging task to get reliable forecasting results from a dynamical weather model. Nevertheless, it is challenging to develop and select an accurate time-series prediction model because it involves training various distinct models to find the best among them. In addition, selecting an optimum topology for the selected models is important too. The accurate forecasting of maximum temperature plays a vital role in human life as well as many sectors such as agriculture and industry. The increase in temperature will deteriorate the highland urban heat, especially in summer, and have a significant influence on people’s health. We applied meta-learning principles to optimize the deep learning network structure for hyperparameter optimization. In particular, the genetic algorithm (GA) for meta-learning was used to select the optimum architecture for the network used. The dataset was used to train and test three different models, namely the artificial neural network (ANN), recurrent neural network (RNN), and long short-term memory (LSTM). Our results demonstrate that the hybrid model of an LSTM network and GA outperforms other models for the long lead time forecasting. Specifically, LSTM forecasts have superiority over RNN and ANN for 15-day-ahead in summer with the root mean square error (RMSE) value of 2.719 (°C).

Download Full-text