scholarly journals PRESEE: An MDL/MML Algorithm to Time-Series Stream Segmenting

2013 ◽  
Vol 2013 ◽  
pp. 1-11 ◽  
Author(s):  
Kaikuo Xu ◽  
Yexi Jiang ◽  
Mingjie Tang ◽  
Changan Yuan ◽  
Changjie Tang
Keyword(s):  
Time Series ◽  
Real Time ◽  
Processing Speed ◽  
Data Stream ◽  
Minimum Description Length ◽  
Data Types ◽  
Minimum Message Length ◽  
Message Length ◽  
Different Types ◽  
State Of Art

Time-series stream is one of the most common data types in data mining field. It is prevalent in fields such as stock market, ecology, and medical care. Segmentation is a key step to accelerate the processing speed of time-series stream mining. Previous algorithms for segmenting mainly focused on the issue of ameliorating precision instead of paying much attention to the efficiency. Moreover, the performance of these algorithms depends heavily on parameters, which are hard for the users to set. In this paper, we proposePRESEE(parameter-free, real-time, and scalable time-series stream segmenting algorithm), which greatly improves the efficiency of time-series stream segmenting. PRESEE is based on both MDL (minimum description length) and MML (minimum message length) methods, which could segment the data automatically. To evaluate the performance of PRESEE, we conduct several experiments on time-series streams of different types and compare it with the state-of-art algorithm. The empirical results show that PRESEE is very efficient for real-time stream datasets by improving segmenting speed nearly ten times. The novelty of this algorithm is further demonstrated by the application of PRESEE in segmenting real-time stream datasets from ChinaFLUX sensor networks data stream.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1601
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Environmental Science ◽  
Short Term Memory ◽  
Arima Model ◽  
Theoretic Approach ◽  
Arma Models ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Real World Datasets

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered. We also develop a simple MML ARIMA model.

scholarly journals Heterogeneous Graphical Granger Causality by Minimum Message Length

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1400
Author(s):  
Kateřina Hlaváčková-Schindler ◽  
Claudia Plant
Keyword(s):  
Time Series ◽  
Causal Inference ◽  
Granger Causality ◽  
Real Data ◽  
Maximum Likelihood Estimates ◽  
Target Time ◽  
Minimum Message Length ◽  
Message Length ◽  
Comparison Methods ◽  
Statistical Model Selection

The heterogeneous graphical Granger model (HGGM) for causal inference among processes with distributions from an exponential family is efficient in scenarios when the number of time observations is much greater than the number of time series, normally by several orders of magnitude. However, in the case of “short” time series, the inference in HGGM often suffers from overestimation. To remedy this, we use the minimum message length principle (MML) to determinate the causal connections in the HGGM. The minimum message length as a Bayesian information-theoretic method for statistical model selection applies Occam’s razor in the following way: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct. Based on the dispersion coefficient of the target time series and on the initial maximum likelihood estimates of the regression coefficients, we propose a minimum message length criterion to select the subset of causally connected time series with each target time series and derive its form for various exponential distributions. We propose two algorithms—the genetic-type algorithm (HMMLGA) and exHMML to find the subset. We demonstrated the superiority of both algorithms in synthetic experiments with respect to the comparison methods Lingam, HGGM and statistical framework Granger causality (SFGC). In the real data experiments, we used the methods to discriminate between pregnancy and labor phase using electrohysterogram data of Islandic mothers from Physionet databasis. We further analysed the Austrian climatological time measurements and their temporal interactions in rain and sunny days scenarios. In both experiments, the results of HMMLGA had the most realistic interpretation with respect to the comparison methods. We provide our code in Matlab. To our best knowledge, this is the first work using the MML principle for causal inference in HGGM.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Real-time interpolation of streaming data

2020 ◽  
Vol 21 (4) ◽  
Author(s):  
Roman Dębski
Keyword(s):  
Time Series ◽  
Finite Difference ◽  
Real Time ◽  
Performance Measures ◽  
Spline Interpolation ◽  
Streaming Data ◽  
Data Sets ◽  
Look Ahead ◽  
Different Types ◽  
Hermite Splines

One of the key elements of real-time $C^1$-continuous cubic spline interpolation of streaming data is an estimator of the first derivative of the interpolated function that is more accurate than the ones based on finite difference schemas.Two such greedy look-ahead heuristic estimators (denoted as MinBE and MinAJ2) based on Calculus of Variations are formally defined (in closed form) together with the corresponding cubic splines they generate, and then comparatively evaluated in a series of numerical experiments involving different types of performance measures. The results presented show that the cubic Hermite splines generated by heuristic MinAJ2 significantly outperformed these based on finite difference schemas in terms of all tested performance measures (including convergence).The proposed approach is quite general. It can be directly applied to streams of univariate functional data like time-series. Multidimensional curves defined parametrically, after splitting, can be handled as well. The streaming character of the algorithm means that it can also be useful in processing data sets that are too large to fit in memory (e.g., edge computing devices, embedded time-series databases).

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

Sign in / Sign up

Export Citation Format

Share Document