The Ensemble of Unsupervised Incremental Learning Algorithm for Time Series Data

Based on the consideration of complementary advantages, different wavelet, fractal and statistical methods are integrated to complete the classification feature extraction of time series. Combined with the advantage of process neural networks that processing time-varying information, we propose a fusion classifier with process neural network oriented time series. Be taking advantage of the multi-fractal processing nonlinear feature of time series data classification, the strong adaptability of the wavelet technique for time series data and the effect of statistical features on the classification of time series data, we can achieve the classification feature extraction of time series. Additionally, using time-varying input characteristics of process neural networks, the pattern matching of timevarying input information and space-time aggregation operation is realized. The feature extraction of time series with the above three methods is fused to the distance calculation between time-varying inputs and cluster space in process neural networks. We provide the process neural network fusion to the learning algorithm and optimize the calculation process of the time series classifier. Finally, we report the performance of our classification method using Synthetic Control Charts data from the UCI dataset and illustrate the advantage and validity of the proposed method.

Download Full-text

Deep Temporal Convolution Network for Time Series Classification

Sensors ◽

10.3390/s21020603 ◽

2021 ◽

Vol 21 (2) ◽

pp. 603

Author(s):

Bee Hock David Koh ◽

Chin Leng Peter Lim ◽

Hasnae Rahimi ◽

Wai Lok Woo ◽

Bin Gao

Keyword(s):

Time Series ◽

Time Series Data ◽

Learning Algorithm ◽

Multivariate Time Series ◽

Classification Performance ◽

Series Data ◽

Temporal Context ◽

Complex Data ◽

Layer By Layer ◽

The Right

A neural network that matches with a complex data function is likely to boost the classification performance as it is able to learn the useful aspect of the highly varying data. In this work, the temporal context of the time series data is chosen as the useful aspect of the data that is passed through the network for learning. By exploiting the compositional locality of the time series data at each level of the network, shift-invariant features can be extracted layer by layer at different time scales. The temporal context is made available to the deeper layers of the network by a set of data processing operations based on the concatenation operation. A matching learning algorithm for the revised network is described in this paper. It uses gradient routing in the backpropagation path. The framework as proposed in this work attains better generalization without overfitting the network to the data, as the weights can be pretrained appropriately. It can be used end-to-end with multivariate time series data in their raw form, without the need for manual feature crafting or data transformation. Data experiments with electroencephalogram signals and human activity signals show that with the right amount of concatenation in the deeper layers of the proposed network, it can improve the performance in signal classification.

Download Full-text

Predicting machine failure using recurrent neural network-gated recurrent unit (RNN-GRU) through time series data

Bulletin of Electrical Engineering and Informatics ◽

10.11591/eei.v10i2.2036 ◽

2021 ◽

Vol 10 (2) ◽

pp. 870-878

Author(s):

Zainuddin Z. ◽

P. Akhir E. A. ◽

Hasan M. H.

Keyword(s):

Neural Network ◽

Time Series ◽

Recurrent Neural Network ◽

Time Series Data ◽

Short Term Memory ◽

Learning Algorithm ◽

The State ◽

Series Data ◽

Machine Failure ◽

Gated Recurrent Unit

Time series data often involves big size environment that lead to high dimensionality problem. Many industries are generating time series data that continuously update each second. The arising of machine learning may help in managing the data. It can forecast future instance while handling large data issues. Forecasting is related to predicting task of an upcoming event to avoid any circumstances happen in current environment. It helps those sectors such as production to foresee the state of machine in line with saving the cost from sudden breakdown as unplanned machine failure can disrupt the operation and loss up to millions. Thus, this paper offers a deep learning algorithm named recurrent neural network-gated recurrent unit (RNN-GRU) to forecast the state of machines producing the time series data in an oil and gas sector. RNN-GRU is an affiliation of recurrent neural network (RNN) that can control consecutive data due to the existence of update and reset gates. The gates decided on the necessary information to be kept in the memory. RNN-GRU is a simpler structure of long short-term memory (RNN-LSTM) with 87% of accuracy on prediction.

Download Full-text

Adaptive Synapse Arrangement in Cortical Learning Algorithm

Journal of Advanced Computational Intelligence and Intelligent Informatics ◽

10.20965/jaciii.2021.p0450 ◽

2021 ◽

Vol 25 (4) ◽

pp. 450-466

Author(s):

Takeru Aoki ◽

◽

Keiki Takadama ◽

Hiroyuki Sato

Keyword(s):

New York ◽

Time Series ◽

Prediction Accuracy ◽

Input Data ◽

Time Series Data ◽

Learning Algorithm ◽

Prediction Method ◽

Series Data ◽

Load Prediction ◽

Data Prediction

The cortical learning algorithm (CLA) is a time-series data prediction method that is designed based on the human neocortex. The CLA has multiple columns that are associated with the input data bits by synapses. The input data is then converted into an internal column representation based on the synapse relation. Because the synapse relation between the columns and input data bits is fixed during the entire prediction process in the conventional CLA, it cannot adapt to input data biases. Consequently, columns not used for internal representations arise, resulting in a low prediction accuracy in the conventional CLA. To improve the prediction accuracy of the CLA, we propose a CLA that self-adaptively arranges the column synapses according to the input data tendencies and verify its effectiveness with several artificial time-series data and real-world electricity load prediction data from New York City. Experimental results show that the proposed CLA achieves higher prediction accuracy than the conventional CLA and LSTMs with different network optimization algorithms by arranging column synapses according to the input data tendency.

Download Full-text

Loss-Driven Adversarial Ensemble Deep Learning for On-Line Time Series Analysis

Sustainability ◽

10.3390/su11123489 ◽

2019 ◽

Vol 11 (12) ◽

pp. 3489

Author(s):

Hyungjin Ko ◽

Jaewook Lee ◽

Junyoung Byun ◽

Bumho Son ◽

Saerom Park

Keyword(s):

Time Series ◽

Deep Learning ◽

Time Series Data ◽

Learning Algorithm ◽

Data Distribution ◽

Series Data ◽

Learning Models ◽

Classification Problems ◽

Ensemble Models ◽

On Line

Developing a robust and sustainable system is an important problem in which deep learning models are used in real-world applications. Ensemble methods combine diverse models to improve performance and achieve robustness. The analysis of time series data requires dealing with continuously incoming instances; however, most ensemble models suffer when adapting to a change in data distribution. Therefore, we propose an on-line ensemble deep learning algorithm that aggregates deep learning models and adjusts the ensemble weight based on loss value in this study. We theoretically demonstrate that the ensemble weight converges to the limiting distribution, and, thus, minimizes the average total loss from a new regret measure based on adversarial assumption. We also present an overall framework that can be applied to analyze time series. In the experiments, we focused on the on-line phase, in which the ensemble models predict the binary class for the simulated data and the financial and non-financial real data. The proposed method outperformed other ensemble approaches. Moreover, our method was not only robust to the intentional attacks but also sustainable in data distribution changes. In the future, our algorithm can be extended to regression and multiclass classification problems.

Download Full-text

A machine learning approach for the spatiotemporal forecasting of ecological phenomena using dates of species occurrence records

10.1101/435289 ◽

2018 ◽

Author(s):

César Capinha

Keyword(s):

Machine Learning ◽

Time Series ◽

Environmental Conditions ◽

Time Series Data ◽

Learning Algorithm ◽

Series Data ◽

List Type ◽

Species Occurrence ◽

Mushroom Species ◽

Occurrence Records

AbstractSpatiotemporal forecasts of ecological phenomena are highly useful and significant in scientific and socio-economic applications. Nevertheless, developing the correlative models to make these forecasts is often stalled by the inadequate availability of the ecological time-series data. On the contrary, considerable amounts of temporally discrete biological records are being stored in public databases, and often include the sites and dates of the observation. While these data are reasonably suitable for the development of spatiotemporal forecast models, this possibility remains mostly untested.In this paper, we test an approach to develop spatiotemporal forecasts based on the dates and locations found in species occurrence records. This approach is based on ‘time-series classification’, a field of machine learning, and involves the application of a machine-learning algorithm to classify between time-series representing the environmental conditions that precede the occurrence records and time-series representing other environmental conditions, such as those that generally occur in the sites of the records. We employed this framework to predict the timing of emergence of fruiting bodies of two mushroom species (Boletus edulis and Macrolepiota procera) in countries of Europe, from 2009 to 2015. We compared the predictions from this approach with those from a ‘null’ model, based on the calendar dates of the records.Forecasts made from the environmental-based approach were consistently superior to those drawn from the date-based approach, averaging an area under the receiver operating characteristic curve (AUC) of 0.9 for B. edulis and 0.88 for M. procera, compared to an average AUC of 0.83 achieved by the null models for both species. Prediction errors were distributed across the study area and along the years, lending support to the spatiotemporal representativeness of the values of accuracy measured.Our approach, based on species occurrence records, was able to provide useful forecasts of the timing of emergence of two mushroom species across Europe. Given the increased availability and information contained in this type of records, particularly those supplemented with photographs, the range of events that could be possible to forecast is vast.

Download Full-text

An Evolutionary Functional Link Neural Fuzzy Model for Financial Time Series Forecasting

Modeling Applications and Theoretical Innovations in Interdisciplinary Evolutionary Computation ◽

10.4018/978-1-4666-3628-6.ch012 ◽

2013 ◽

pp. 189-205

Author(s):

S. Chakravarty ◽

P. K. Dash ◽

V. Ravikumar Pandi ◽

B. K. Panigrahi

Keyword(s):

Time Series ◽

Time Series Data ◽

Learning Algorithm ◽

Financial Time Series ◽

Back Propagation ◽

Fuzzy Model ◽

Series Data ◽

Functional Link ◽

Financial Time ◽

Neural Fuzzy

This paper proposes a hybrid model, evolutionary functional link neural fuzzy model (EFLNF), to forecast financial time series where the parameters are optimized by two most efficient evolutionary algorithms: (a) genetic algorithm (GA) and (b) particle swarm optimization (PSO). When the periodicity is just one day, PSO produces a better result than that of GA. But the gap in the performance between them increases as periodicity increases. The convergence speed is also better in case of PSO for one week and one month a head prediction. To testify the superiority of the EFLNF, a number of comparative studies have been made. First, functional link artificial neural network (FLANN) and functional link neural fuzzy (FLNF) were combined with back propagation (BP) learning algorithm. The result shows that FLNF performs better than FLANN. Again, FLNF is compared with EFLNF where the latter outperforms the former irrespective of the periodicity or the learning algorithms with which it has been combined. All models are used to predict the most chaotic financial time series data; BSE Sensex and S&P CNX Nifty stock indices one day, one week and one month in advance.

Download Full-text

Atmospheric Correction Thresholds for Ground-Based Radar Interferometry Deformation Monitoring Estimated Using Time Series Analyses

Remote Sensing ◽

10.3390/rs12142236 ◽

2020 ◽

Vol 12 (14) ◽

pp. 2236 ◽

Cited By ~ 1

Author(s):

Tomasz Owerko ◽

Przemysław Kuras ◽

Łukasz Ortyl

Keyword(s):

Time Series ◽

Satellite Data ◽

Time Series Data ◽

Learning Algorithm ◽

Atmospheric Correction ◽

Deformation Monitoring ◽

Radar Interferometry ◽

Series Data ◽

Atmospheric Conditions ◽

The Stability

Ground-based radar interferometry (GBSAR) is a useful method to control the stability of engineering objects and elements of geographical spaces at risk of deformation or displacement. To secure accurate and credible measurement results, it is crucial to consider atmospheric conditions as they influence the corrections to distance measurements. These conditions are especially important considering the radar bandwidth used. Measurements for the stability of engineering objects are not always performed in locations where meteorological monitoring is prevalent; however, information about the range of variability in atmospheric corrections is always welcome. The authors present a hybrid method to estimate the probable need of atmospheric corrections, which allows partly eliminating false positive alarms of deformations as caused by atmospheric fluctuations. Unlike the numerous publications on atmospheric reductions focused on the current state of the atmosphere, the proposed solution is based on applying a classic machine learning algorithm designed for the SARIMAX (Seasonal Autoregressive Integrated Moving Average with covariate at time) time series data model for satellite data shared by NASA (National Aeronautics and Space Administration) during the Landsat MODIS (Moderate Resolution Imaging Spectroradiometer) mission before performing residual estimation during the monitoring phase. Example calculations (proof of concept) were made for ten-year satellite data covering a region for experimental flood bank stability observations as performed using the IBIS-L (Image by Interferometric Survey—Landslide) radar and for target monitoring data (ground measurements).

Download Full-text

Adaptive forgetting factor OS-ELM and bootstrap for time series prediction

International Journal of Modeling Simulation and Scientific Computing ◽

10.1142/s1793962317500295 ◽

2017 ◽

Vol 08 (03) ◽

pp. 1750029 ◽

Cited By ~ 2

Author(s):

Jingzhong Liu

Keyword(s):

Time Series ◽

Time Series Data ◽

Learning Algorithm ◽

Prediction Interval ◽

Time Series Prediction ◽

Training Data ◽

Series Data ◽

Forgetting Factor ◽

Adaptive Forgetting Factor ◽

Adaptive Forgetting

Online sequential extreme learning machine (OS-ELM) for single-hidden layer feedforward networks (SLFNs) is an effective machine learning algorithm. But OS-ELM has some underlying weaknesses of neglecting time series timeliness and being incapable to provide quantitative uncertainty for prediction. To overcome these shortcomings, a time series prediction method is presented based on the combination of OS-ELM with adaptive forgetting factor (AFF-OS-ELM) and bootstrap (B-AFF-OS-ELM). Firstly, adaptive forgetting factor is added into OS-ELM for adjusting the effective window length of training data during OS-ELM sequential learning phase. Secondly, the current bootstrap is developed to fit time series prediction online. Then associated with improved bootstrap, the proposed method can compute prediction interval as uncertainty information, meanwhile the improved bootstrap enhances prediction accuracy and stability of AFF-OS-ELM. Performances of B-AFF-OS-ELM are benchmarked with other traditional and improved OS-ELM on simulation and practical time series data. Results indicate the significant performances achieved by B-AFF-OS-ELM.

Download Full-text

Application of the NARX neural network for predicting a one-dimensional time series

Eastern-European Journal of Enterprise Technologies ◽

10.15587/1729-4061.2021.242442 ◽

2021 ◽

Vol 5 (4 (113)) ◽

pp. 12-19

Author(s):

Tansaule Serikov ◽

Ainur Zhetpisbayeva ◽

Sharafat Mirzakulova ◽

Kairatbek Zhetpisbayev ◽

Zhanar Ibrayeva ◽

...

Keyword(s):

Neural Network ◽

Time Series ◽

Initial Data ◽

Network Traffic ◽

Time Series Data ◽

Network Flows ◽

Learning Algorithm ◽

Complex Structure ◽

Series Data ◽

Packet Arrival

Time series data analysis and forecasting tool for studying the data on the use of network traffic is very important to provide acceptable and good quality network services, including network monitoring, resource management, and threat detection. More and more, the behavior of network traffic is described by the theory of deterministic chaos. The traffic of a modern network has a complex structure, an uneven rate of packet arrival for service by network devices. Predicting network traffic is still an important task, as forecast data provide the necessary information to solve the problem of managing network flows. Numerous studies of actually measured data confirm that they are nonstationary and their structure is multicomponent. This paper presents modeling using Nonlinear Autoregression Exogenous (NARX) algorithm for predicting network traffic datasets. NARX is one of the models that can be used to demonstrate non-linear systems, especially in modeling time series datasets. In other words, they called the categories of dynamic feedback networks covering several layers of the network. An artificial neural network (ANN) was developed, trained and tested using the LM learning algorithm (Levenberg-Macwardt). The initial data for the prediction is the actual measured network traffic of the packet rate. As a result of the study of the initial data, the best value of the smallest mean-square error MSE (Mean Squared Error) was obtained with the epoch value equal to 18. As for the regression R, its output ANN values in relation to the target for training, validation and testing were 0.97743. 0.9638 and 0.94907, respectively, with an overall regression value of 0.97134, which ensures that all datasets match exactly. Experimental results (MSE, R) have proven the method's ability to accurately estimate and predict network traffic

Download Full-text