scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1601
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Environmental Science ◽  
Short Term Memory ◽  
Arima Model ◽  
Theoretic Approach ◽  
Arma Models ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Real World Datasets

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered. We also develop a simple MML ARIMA model.

Forecasting container throughput with long short-term memory networks

2019 ◽  
Vol 120 (3) ◽  
pp. 425-441 ◽  
Author(s):  
Sonali Shankar ◽  
P. Vigneswara Ilavarasan ◽  
Sushil Punia ◽  
Surya Prakash Singh
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Short Term ◽  
Error Matrix ◽  
Term Memory ◽  
Content Type ◽  
Forecasting Performance ◽  
Long Short Term Memory

Purpose Better forecasting always leads to better management and planning of the operations. The container throughput data are complex and often have multiple seasonality. This makes it difficult to forecast accurately. The purpose of this paper is to forecast container throughput using deep learning methods and benchmark its performance over other traditional time-series methods. Design/methodology/approach In this study, long short-term memory (LSTM) networks are implemented to forecast container throughput. The container throughput data of the Port of Singapore are used for empirical analysis. The forecasting performance of the LSTM model is compared with seven different time-series forecasting methods, namely, autoregressive integrated moving average (ARIMA), simple exponential smoothing, Holt–Winter’s, error-trend-seasonality, trigonometric regressors (TBATS), neural network (NN) and ARIMA + NN. The relative error matrix is used to analyze the performance of the different models with respect to bias, accuracy and uncertainty. Findings The results showed that LSTM outperformed all other benchmark methods. From a statistical perspective, the Diebold–Mariano test is also conducted to further substantiate better forecasting performance of LSTM over other counterpart methods. Originality/value The proposed study is a contribution to the literature on the container throughput forecasting and adds value to the supply chain theory of forecasting. Second, this study explained the architecture of the deep-learning-based LSTM method and discussed in detail the steps to implement it.

scholarly journals The Deep Learning LSTM and MTD Models Best Predict Acute Respiratory Infection among Under-Five-Year Old Children in Somaliland

Symmetry ◽  
2021 ◽  
Vol 13 (7) ◽  
pp. 1156
Author(s):  
Mohamed Yusuf Hassan
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Respiratory Infection ◽  
Acute Respiratory Infection ◽  
Short Term Memory ◽  
Mean Deviation ◽  
Sarima Model ◽  
Under Five ◽  
Competing Models ◽  
Long Short Term Memory

The most effective techniques for predicting time series patterns include machine learning and classical time series methods. The aim of this study is to search for the best artificial intelligence and classical forecasting techniques that can predict the spread of acute respiratory infection (ARI) and pneumonia among under-five-year old children in Somaliland. The techniques used in the study include seasonal autoregressive integrated moving averages (SARIMA), mixture transitions distribution (MTD), and long short term memory (LSTM) deep learning. The data used in the study were monthly observations collected from five regions in Somaliland from 2011–2014. Prediction results from the three best competing models are compared by using root mean square error (RMSE) and absolute mean deviation (MAD) accuracy measures. Results have shown that the deep learning LSTM and MTD models slightly outperformed the classical SARIMA model in predicting ARI values.

Prognostics and RUL Estimations of SAC305, SAC105 and SnAg Solders Under Temperature and Vibration Using Long Short-Term Memory (LSTM) Deep Learning

2021 ◽  
Author(s):  
Pradeep Lall ◽  
Tony Thomas ◽  
Ken Blecker
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Remaining Useful Life ◽  
Operating Conditions ◽  
Short Term ◽  
Term Memory ◽  
Rectangular Pattern ◽  
Useful Life ◽  
Long Short Term Memory

Abstract Prognostics and Remaining Useful Life (RUL) estimations of complex systems are essential to operational safety, increased efficiency, and help to schedule maintenance proactively. Modeling the remaining useful life of a system with many complexities is possible with the rapid development in the field of deep learning as a computational technique for failure prediction. Deep learning can adapt to multivariate parameters complex and nonlinear behavior, which is difficult using traditional time-series models for forecasting and prediction purposes. In this paper, a deep learning approach based on Long Short-Term Memory (LSTM) network is used to predict the remaining useful life of the PCB at different conditions of temperature and vibration. This technique can identify the different underlying patterns in the time series that can predict the RUL. This study involves feature vector identification and RUL estimations for SAC305, SAC105, and Tin Lead solder PCBs under different vibration levels and temperature conditions. The acceleration levels of vibration are fixed at 5g and 10g, while the temperature levels are 55°C and 100°C. The test board is a multilayer FR4 configuration with JEDEC standard dimensions consists of twelve packages arranged in a rectangular pattern. Strain signals are acquired from the backside of the PCB at symmetric locations to identify the failure of all the packages during vibration. The strain signals are resistance values that are acquired simultaneously during the experiment until the failure of most of the packages on the board. The feature vectors are identified from statistical analysis on the strain signals frequency and instantaneous frequency components. The principal component analysis is used as a data reduction technique to identify the different patterns produced from the four strain signals with failures of the packages during vibration. LSTM deep learning method is used to model the RUL of the packages at different individual operating conditions of vibration for all three solder materials involved in this study. A combined model for RUL prediction for a material that can take care of the changes in the operating conditions is also modeled for each material.

scholarly journals An Improved Deep Learning Algorithm for Risk Prediction of Corporate Internet Reporting

2020 ◽  
Vol 34 (4) ◽  
pp. 437-444
Author(s):  
Lingyan Ou ◽  
Ling Chen
Keyword(s):  
Deep Learning ◽  
Risk Prediction ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
Moving Average ◽  
Arma Model ◽  
Evaluation Index System ◽  
Online Information ◽  
Deep Learning Algorithm ◽  
Research Findings

Corporate internet reporting (CIR) has such advantages as the strong timeliness, large amount, and wide coverage of financial information. However, the CIR, like any other online information, faces various risks. With the aid of the increasingly sophisticated artificial intelligence (AI) technology, this paper proposes an improved deep learning algorithm for the prediction of CIR risks, aiming to improve the accuracy of CIR risk prediction. After building a reasonable evaluation index system (EIS) for CIR risks, the data involved in risk rating and the prediction of risk transmission effect (RTE) were subject to structured feature extraction and time series construction. Next, a combinatory CIR risk prediction model was established by combining the autoregressive moving average (ARMA) model with long short-term memory (LSTM). The former is good at depicting linear series, and the latter excels in describing nonlinear series. Experimental results demonstrate the effectiveness of the ARMA-LSTM model. The research findings provide a good reference for applying AI technology in risk prediction of other areas.

scholarly journals Deep Learning with Long Short-Term Memory for Time Series Prediction

2019 ◽  
Vol 57 (6) ◽  
pp. 114-119 ◽  
Author(s):  
Yuxiu Hua ◽  
Zhifeng Zhao ◽  
Rongpeng Li ◽  
Xianfu Chen ◽  
Zhiming Liu ◽  
...  
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Time Series Prediction ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

scholarly journals A Hybrid Approach for Turning Intention Prediction Based on Time Series Forecasting and Deep Learning

Sensors ◽  
2020 ◽  
Vol 20 (17) ◽  
pp. 4887
Author(s):  
Hailun Zhang ◽  
Rui Fu
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Hybrid Approach ◽  
Lateral Acceleration ◽  
Short Term ◽  
Lateral Velocity ◽  
Term Memory ◽  
Turning Behavior ◽  
Long Short Term Memory

At an intersection with complex traffic flow, the early detection of the intention of drivers in surrounding vehicles can enable advanced driver assistance systems (ADAS) to warn the driver in advance or prompt its subsystems to assess the risk and intervene early. Although different drivers show various driving characteristics, the kinematic parameters of human-driven vehicles can be used as a predictor for predicting the driver’s intention within a short time. In this paper, we propose a new hybrid approach for vehicle behavior recognition at intersections based on time series prediction and deep learning networks. First, the lateral position, longitudinal position, speed, and acceleration of the vehicle are predicted using the online autoregressive integrated moving average (ARIMA) algorithm. Next, a variant of the long short-term memory network, called the bidirectional long short-term memory (Bi-LSTM) network, is used to detect the vehicle’s turning behavior using the predicted parameters, as well as the derived parameters, i.e., the lateral velocity, lateral acceleration, and heading angle. The validity of the proposed method is verified at real intersections using the public driving data of the next generation simulation (NGSIM) project. The results of the turning behavior detection show that the proposed hybrid approach exhibits significant improvement over a conventional algorithm; the average recognition rates are 94.2% and 93.5% at 2 s and 1 s, respectively, before initiating the turning maneuver.

scholarly journals A Fuzzy Set-Valued Autoregressive Moving Average Model and Its Applications

Symmetry ◽  
2018 ◽  
Vol 10 (8) ◽  
pp. 324 ◽  
Author(s):  
Dabuxilatu Wang ◽  
Liang Zhang
Keyword(s):  
Time Series ◽  
Time Series Analysis ◽  
Empirical Analysis ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Complex Data ◽  
Arma Models ◽  
Linguistic Data ◽  
Series Analysis

Autoregressive moving average (ARMA) models are important in many fields and applications, although they are most widely applied in time series analysis. Expanding the ARMA models to the case of various complex data is arguably one of the more challenging problems in time series analysis and mathematical statistics. In this study, we extended the ARMA model to the case of linguistic data that can be modeled by some symmetric fuzzy sets, and where the relations between the linguistic data of the time series can be considered as the ordinary stochastic correlation rather than fuzzy logical relations. Therefore, the concepts of set-valued or interval-valued random variables can be employed, and the notions of Aumann expectation, Fréchet variance, and covariance, as well as standardized process, were used to construct the ARMA model. We firstly determined that the estimators from the least square estimation of the ARMA (1,1) model under some L2 distance between two sets are weakly consistent. Moreover, the justified linguistic data-valued ARMA model was applied to forecast the linguistic monthly Hang Seng Index (HSI) as an empirical analysis. The obtained results from the empirical analysis indicate that the accuracy of the prediction produced from the proposed model is better than that produced from the classical one-order, two-order, three-order autoregressive (AR(1), AR(2), AR(3)) models, as well as the (1,1)-order autoregressive moving average (ARMA(1,1)) model.

Deep learning-based container throughput forecasting: a triple bottom line approach

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Sonali Shankar ◽  
Sushil Punia ◽  
P. Vigneswara Ilavarasan
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Short Term ◽  
Bottom Line ◽  
Content Type ◽  
Autoregressive Integrated Moving Average ◽  
Forecasting Method ◽  
Memory Network ◽  
Long Short Term Memory

PurposeContainer throughput forecasting plays a pivotal role in strategic, tactical and operational level decision-making. The determination and analysis of the influencing factors of container throughput are observed to enhance the predicting accuracy. Therefore, for effective port planning and management, this study employs a deep learning-based method to forecast the container throughput while considering the influence of economic, environmental and social factors on throughput forecasting.Design/methodology/approachA novel multivariate container throughput forecasting method is proposed using long short-term memory network (LSTM). The external factors influencing container throughput, delineated using triple bottom line, are considered as an input to the forecasting method. The principal component analysis (PCA) is employed to reduce the redundancy of the input variables. The container throughput data of the Port of Los Angeles (PLA) is considered for empirical analysis. The forecasting accuracy of the proposed method is measured via an error matrix. The accuracy of the results is further substantiated by the Diebold-Mariano statistical test.FindingsThe result of the proposed method is benchmarked with vector autoregression (VAR), autoregressive integrated moving average (ARIMAX) and LSTM. It is observed that the proposed method outperforms other counterpart methods. Though PCA was not an integral part of the forecasting process, it facilitated the prediction by means of “less data, more accuracy.”Originality/valueA novel deep learning-based forecasting method is proposed to predict container throughput using a hybridized autoregressive integrated moving average with external factors model and long short-term memory network (ARIMAX-LSTM).

Sign in / Sign up

Export Citation Format

Share Document