scholarly journals Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Electronics ◽  
2019 ◽  
Vol 8 (8) ◽  
pp. 876 ◽  
Author(s):  
Renzhuo Wan ◽  
Shuping Mei ◽  
Jun Wang ◽  
Min Liu ◽  
Fan Yang
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Prediction Accuracy ◽  
Time Series Data ◽  
Short Term Memory ◽  
Time Series Prediction ◽  
Multivariate Time Series ◽  
Series Data ◽  
Asymmetric Structure ◽  
Convolutional Network

Multivariable time series prediction has been widely studied in power energy, aerology, meteorology, finance, transportation, etc. Traditional modeling methods have complex patterns and are inefficient to capture long-term multivariate dependencies of data for desired forecasting accuracy. To address such concerns, various deep learning models based on Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) methods are proposed. To improve the prediction accuracy and minimize the multivariate time series data dependence for aperiodic data, in this article, Beijing PM2.5 and ISO-NE Dataset are analyzed by a novel Multivariate Temporal Convolution Network (M-TCN) model. In this model, multi-variable time series prediction is constructed as a sequence-to-sequence scenario for non-periodic datasets. The multichannel residual blocks in parallel with asymmetric structure based on deep convolution neural network is proposed. The results are compared with rich competitive algorithms of long short term memory (LSTM), convolutional LSTM (ConvLSTM), Temporal Convolution Network (TCN) and Multivariate Attention LSTM-FCN (MALSTM-FCN), which indicate significant improvement of prediction accuracy, robust and generalization of our model.

Neural Network Techniques for Time Series Prediction: A Review

2019 ◽  
Vol 3 (3) ◽  
Author(s):  
Muhammad Faheem Mushtaq ◽  
Urooj Akram ◽  
Muhammad Aamir ◽  
Haseeb Ali ◽  
Muhammad Zulqarnain
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Time Series Data ◽  
Weather Prediction ◽  
Time Series Prediction ◽  
Series Data ◽  
Prediction Problem ◽  
Neural Network Models ◽  
Physical Time

It is important to predict a time series because many problems that are related to prediction such as health prediction problem, climate change prediction problem and weather prediction problem include a time component. To solve the time series prediction problem various techniques have been developed over many years to enhance the accuracy of forecasting. This paper presents a review of the prediction of physical time series applications using the neural network models. Neural Networks (NN) have appeared as an effective tool for forecasting of time series.  Moreover, to resolve the problems related to time series data, there is a need of network with single layer trainable weights that is Higher Order Neural Network (HONN) which can perform nonlinearity mapping of input-output. So, the developers are focusing on HONN that has been recently considered to develop the input representation spaces broadly. The HONN model has the ability of functional mapping which determined through some time series problems and it shows the more benefits as compared to conventional Artificial Neural Networks (ANN). The goal of this research is to present the reader awareness about HONN for physical time series prediction, to highlight some benefits and challenges using HONN.

scholarly journals Research on maintenance spare parts requirement prediction based on LSTM recurrent neural network

Open Physics ◽  
2021 ◽  
Vol 19 (1) ◽  
pp. 618-627
Author(s):  
Weixing Song ◽  
Jingjing Wu ◽  
Jianshe Kang ◽  
Jun Zhang
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Short Term Memory ◽  
Spare Parts ◽  
Performance Comparison ◽  
Series Data ◽  
Generalized Regression Neural Network ◽  
Particle Swarm Algorithm

Abstract The aim of this study was to improve the low accuracy of equipment spare parts requirement predicting, which affects the quality and efficiency of maintenance support, based on the summary and analysis of the existing spare parts requirement predicting research. This article introduces the current latest popular long short-term memory (LSTM) algorithm which has the best effect on time series data processing to equipment spare parts requirement predicting, according to the time series characteristics of spare parts consumption data. A method for predicting the requirement for maintenance spare parts based on the LSTM recurrent neural network is proposed, and the network structure is designed in detail, the realization of network training and network prediction is given. The advantages of particle swarm algorithm are introduced to optimize the network parameters, and actual data of three types of equipment spare parts consumption are used for experiments. The performance comparison of predictive models such as BP neural network, generalized regression neural network, wavelet neural network, and squeeze-and-excitation network prove that the new method is effective and provides an effective method for scientifically predicting the requirement for maintenance spare parts and improving the quality of equipment maintenance.

scholarly journals Long Short Term Memory Recurrent Network for Standard and Poor’s 500 Index Modelling

2018 ◽  
Vol 7 (4.15) ◽  
pp. 25 ◽  
Author(s):  
Said Jadid Abdulkadir ◽  
Hitham Alhussian ◽  
Muhammad Nazmi ◽  
Asim A Elsheikh
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Short Term Memory ◽  
Series Data ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Forecasting time-series data are imperative especially when planning is required through modelling using uncertain knowledge of future events. Recurrent neural network models have been applied in the industry and outperform standard artificial neural networks in forecasting, but fail in long term time-series forecasting due to the vanishing gradient problem. This study offers a robust solution that can be implemented for long-term forecasting using a special architecture of recurrent neural network known as Long Short Term Memory (LSTM) model to overcome the vanishing gradient problem. LSTM is specially designed to avoid the long-term dependency problem as their default behavior. Empirical analysis is performed using quantitative forecasting metrics and comparative model performance on the forecasted outputs. An evaluation analysis is performed to validate that the LSTM model provides better forecasted outputs on Standard & Poor’s 500 Index (S&P 500) in terms of error metrics as compared to other forecasting models.  

GRU-corr Neural Network Optimized by Improved PSO Algorithm for Time Series Prediction

2020 ◽  
Vol 29 (07n08) ◽  
pp. 2040010
Author(s):  
Shao-Pei Ji ◽  
Yu-Long Meng ◽  
Liang Yan ◽  
Gui-Shan Dong ◽  
Dong Liu
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Time Series Data ◽  
Time Series Prediction ◽  
Prediction Method ◽  
Pso Algorithm ◽  
Series Data ◽  
Multi Scale ◽  
Network Parameters ◽  
Angle Data

Time series data from real problems have nonlinear, non-smooth, and multi-scale composite characteristics. This paper first proposes a gated recurrent unit-correction (GRU-corr) network model, which adds a correction layer to the GRU neural network. Then, a adaptive staged variation PSO (ASPSO) is proposed. Finally, to overcome the drawbacks of the imprecise selection of the GRU-corr network parameters and obtain the high-precision global optimization of network parameters, weight parameters and the hidden nodes number of GRU-corr is optimized by ASPSO, and a time series prediction model (ASPSO-GRU-corr) is proposed based on the GRU-corr optimized by ASPSO. In the experiment, a comparative analysis of the optimization performance of ASPSO on a benchmark function was performed to verify its validity, and then the ASPSO-GRU-corr model is used to predict the ship motion cross-sway angle data. The results show that, ASPSO has better optimization performance and convergence speed compared with other algorithms, while the ASPSO-GRU-corr has higher generalization performance and lower architecture complexity. The ASPSO-GRU-corr can reveal the intrinsic multi-scale composite features of the time series, which is a reliable nonlinear and non-steady time series prediction method.

scholarly journals STL-ATTLSTM: Vegetable Price Forecasting Using STL and Attention Mechanism-Based LSTM

Agriculture ◽  
2020 ◽  
Vol 10 (12) ◽  
pp. 612
Author(s):  
Helin Yin ◽  
Dong Jin ◽  
Yeong Hyeon Gu ◽  
Chang Jin Park ◽  
Sang Keun Han ◽  
...  
Keyword(s):  
Time Series ◽  
Prediction Accuracy ◽  
Crop Production ◽  
Time Series Data ◽  
Short Term Memory ◽  
Attention Mechanism ◽  
Series Data ◽  
Percentage Error ◽  
Hot Pepper ◽  
Types Of Information

It is difficult to forecast vegetable prices because they are affected by numerous factors, such as weather and crop production, and the time-series data have strong non-linear and non-stationary characteristics. To address these issues, we propose the STL-ATTLSTM (STL-Attention-based LSTM) model, which integrates the seasonal trend decomposition using the Loess (STL) preprocessing method and attention mechanism based on long short-term memory (LSTM). The proposed STL-ATTLSTM forecasts monthly vegetable prices using various types of information, such as vegetable prices, weather information of the main production areas, and market trading volumes. The STL method decomposes time-series vegetable price data into trend, seasonality, and remainder components. It uses the remainder component by removing the trend and seasonality components. In the model training process, attention weights are assigned to all input variables; thus, the model’s prediction performance is improved by focusing on the variables that affect the prediction results. The proposed STL-ATTLSTM was applied to five crops, namely cabbage, radish, onion, hot pepper, and garlic, and its performance was compared to three benchmark models (i.e., LSTM, attention LSTM, and STL-LSTM). The performance results show that the LSTM model combined with the STL method (STL-LSTM) achieved a 12% higher prediction accuracy than the attention LSTM model that did not use the STL method and solved the prediction lag arising from high seasonality. The attention LSTM model improved the prediction accuracy by approximately 4% to 5% compared to the LSTM model. The STL-ATTLSTM model achieved the best performance, with an average root mean square error (RMSE) of 380, and an average mean absolute percentage error (MAPE) of 7%.

scholarly journals A hybrid elman neural network predictor for time series prediction

2018 ◽  
Vol 7 (2.20) ◽  
pp. 159 ◽  
Author(s):  
N Mohana Sundaram ◽  
S N. Sivanandam
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Time Delay ◽  
Adaptive Learning ◽  
Time Series Data ◽  
Time Series Prediction ◽  
Series Data ◽  
Elman Neural Network ◽  
Dynamic Processing ◽  
Elman Recurrent Neural Network

Artificial Neural Networks have become popular in the world of prediction and forecasting due to their nonlinear nonparametric adaptive-learning property. They become an important tool in data analysis and data mining applications. Elman neural network due to its recurrent nature and dynamic processing capabilities can perform the prediction process with a good range of accuracy. In this paper an Elman recurrent Neural Network is hybridised with a time delay called a tap delay line for time series prediction process to improve its performance. The Elman neural network with the time delay inputs is trained tested and validated using the solar sun spot time series data that contains the monthly mean sunspot numbers for a 240 year period having 2899 data values. The results confirm that the proposed Elman network hybridised with time delay inputs can predict the time series with more accurately and effectively than the existing methods.  

Variance error of multi-classification based anomaly detection for time series data

2020 ◽  
pp. 1-16
Author(s):  
Baoquan Wang ◽  
Tonghai Jiang ◽  
Xi Zhou ◽  
Bo Ma ◽  
Fan Zhao ◽  
...  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Short Term Memory ◽  
Computational Cost ◽  
Reconstruction Error ◽  
Detection Methods ◽  
Series Data ◽  
Data Set

For abnormal detection of time series data, the supervised anomaly detection methods require labeled data. While the range of outlier factors used by the existing semi-supervised methods varies with data, model and time, the threshold for determining abnormality is difficult to obtain, in addition, the computational cost of the way to calculate outlier factors from other data points in the data set is also very large. These make such methods difficult to practically apply. This paper proposes a framework named LSTM-VE which uses clustering combined with visualization method to roughly label normal data, and then uses the normal data to train long short-term memory (LSTM) neural network for semi-supervised anomaly detection. The variance error (VE) of the normal data category classification probability sequence is used as outlier factor. The framework enables anomaly detection based on deep learning to be practically applied and using VE avoids the shortcomings of existing outlier factors and gains a better performance. In addition, the framework is easy to expand because the LSTM neural network can be replaced with other classification models. Experiments on the labeled and real unlabeled data sets prove that the framework is better than replicator neural networks with reconstruction error (RNN-RS) and has good scalability as well as practicability.

Remaining Useful Life Estimation and Prognostication of SAC305 PCB's for Dynamic Conditions of Temperature and Vibration Loads

2021 ◽  
Author(s):  
Pradeep Lall ◽  
Tony Thomas ◽  
Ken Blecker
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Time Series Data ◽  
Short Term Memory ◽  
Multivariate Time Series ◽  
Remaining Useful Life ◽  
Series Data ◽  
Sac305 Solder ◽  
Feature Vectors ◽  
Useful Life

Abstract This study focuses on the feature vector identification and Remaining Useful Life (RUL) estimation of SAC305 solder alloy PCB's of two different configurations during varying conditions of temperature and vibration. The feature vectors are identified using the strain signals acquired from four symmetrical locations of the PCB at regular intervals during vibration. Two different types of experiments are employed to characterize the PCB's dynamic changes with varying temperature and acceleration levels. The strain signals acquired during each of these experiments are compared based on both time and frequency domain characteristics. Different statistical and frequency-based techniques were used to identify the strain signal variations with changes in the environment and loading conditions. The feature vectors in predicting failure at a constant working temperature and load were identified, and as an extension to this work, the effectiveness of the feature vectors during varying conditions of temperature and acceleration levels are investigated. The remaining Useful Life of the packages was estimated using a deep learning approach based on Long Short Term Memory (LSTM) network. This technique can identify the underlying patterns in multivariate time series data that can predict the packages' life. The autocorrelation function's residuals were used as the multivariate time series data in conjunction with the LSTM deep learning technique to forecast the packages' life at different varying temperatures and acceleration levels during vibration.

Sign in / Sign up

Export Citation Format

Share Document