scholarly journals Heterogeneous Graphical Granger Causality by Minimum Message Length

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1400
Author(s):  
Kateřina Hlaváčková-Schindler ◽  
Claudia Plant
Keyword(s):  
Time Series ◽  
Causal Inference ◽  
Granger Causality ◽  
Real Data ◽  
Maximum Likelihood Estimates ◽  
Target Time ◽  
Minimum Message Length ◽  
Message Length ◽  
Comparison Methods ◽  
Statistical Model Selection

The heterogeneous graphical Granger model (HGGM) for causal inference among processes with distributions from an exponential family is efficient in scenarios when the number of time observations is much greater than the number of time series, normally by several orders of magnitude. However, in the case of “short” time series, the inference in HGGM often suffers from overestimation. To remedy this, we use the minimum message length principle (MML) to determinate the causal connections in the HGGM. The minimum message length as a Bayesian information-theoretic method for statistical model selection applies Occam’s razor in the following way: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct. Based on the dispersion coefficient of the target time series and on the initial maximum likelihood estimates of the regression coefficients, we propose a minimum message length criterion to select the subset of causally connected time series with each target time series and derive its form for various exponential distributions. We propose two algorithms—the genetic-type algorithm (HMMLGA) and exHMML to find the subset. We demonstrated the superiority of both algorithms in synthetic experiments with respect to the comparison methods Lingam, HGGM and statistical framework Granger causality (SFGC). In the real data experiments, we used the methods to discriminate between pregnancy and labor phase using electrohysterogram data of Islandic mothers from Physionet databasis. We further analysed the Austrian climatological time measurements and their temporal interactions in rain and sunny days scenarios. In both experiments, the results of HMMLGA had the most realistic interpretation with respect to the comparison methods. We provide our code in Matlab. To our best knowledge, this is the first work using the MML principle for causal inference in HGGM.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1601
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Environmental Science ◽  
Short Term Memory ◽  
Arima Model ◽  
Theoretic Approach ◽  
Arma Models ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Real World Datasets

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered. We also develop a simple MML ARIMA model.

scholarly journals Causal inference with multiple time series: principles and problems

2013 ◽  
Vol 371 (1997) ◽  
pp. 20110613 ◽  
Author(s):  
Michael Eichler
Keyword(s):  
Time Series ◽  
Causal Inference ◽  
Granger Causality ◽  
Latent Variables ◽  
Time Series Data ◽  
Series Data ◽  
Multiple Time ◽  
Identification Algorithm ◽  
Multiple Time Series ◽  
Theoretical Justification

I review the use of the concept of Granger causality for causal inference from time-series data. First, I give a theoretical justification by relating the concept to other theoretical causality measures. Second, I outline possible problems with spurious causality and approaches to tackle these problems. Finally, I sketch an identification algorithm that learns causal time-series structures in the presence of latent variables. The description of the algorithm is non-technical and thus accessible to applied scientists who are interested in adopting the method.

scholarly journals PRESEE: An MDL/MML Algorithm to Time-Series Stream Segmenting

2013 ◽  
Vol 2013 ◽  
pp. 1-11 ◽  
Author(s):  
Kaikuo Xu ◽  
Yexi Jiang ◽  
Mingjie Tang ◽  
Changan Yuan ◽  
Changjie Tang
Keyword(s):  
Time Series ◽  
Real Time ◽  
Processing Speed ◽  
Data Stream ◽  
Minimum Description Length ◽  
Data Types ◽  
Minimum Message Length ◽  
Message Length ◽  
Different Types ◽  
State Of Art

Time-series stream is one of the most common data types in data mining field. It is prevalent in fields such as stock market, ecology, and medical care. Segmentation is a key step to accelerate the processing speed of time-series stream mining. Previous algorithms for segmenting mainly focused on the issue of ameliorating precision instead of paying much attention to the efficiency. Moreover, the performance of these algorithms depends heavily on parameters, which are hard for the users to set. In this paper, we proposePRESEE(parameter-free, real-time, and scalable time-series stream segmenting algorithm), which greatly improves the efficiency of time-series stream segmenting. PRESEE is based on both MDL (minimum description length) and MML (minimum message length) methods, which could segment the data automatically. To evaluate the performance of PRESEE, we conduct several experiments on time-series streams of different types and compare it with the state-of-art algorithm. The empirical results show that PRESEE is very efficient for real-time stream datasets by improving segmenting speed nearly ten times. The novelty of this algorithm is further demonstrated by the application of PRESEE in segmenting real-time stream datasets from ChinaFLUX sensor networks data stream.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Kumaraswamy Generalized Power Lomax Distributionand Its Applications

Stats ◽  
2021 ◽  
Vol 4 (1) ◽  
pp. 28-45
Author(s):  
Vasili B.V. Nagarjuna ◽  
R. Vishnu Vardhan ◽  
Christophe Chesneau
Keyword(s):  
Hazard Rate ◽  
Real Data ◽  
Rate Function ◽  
Maximum Likelihood Estimates ◽  
Parameter Estimates ◽  
Parameter Distribution ◽  
Data Sets ◽  
Lomax Distribution ◽  
Entropy Measures ◽  
Modeling Behavior

In this paper, a new five-parameter distribution is proposed using the functionalities of the Kumaraswamy generalized family of distributions and the features of the power Lomax distribution. It is named as Kumaraswamy generalized power Lomax distribution. In a first approach, we derive its main probability and reliability functions, with a visualization of its modeling behavior by considering different parameter combinations. As prime quality, the corresponding hazard rate function is very flexible; it possesses decreasing, increasing and inverted (upside-down) bathtub shapes. Also, decreasing-increasing-decreasing shapes are nicely observed. Some important characteristics of the Kumaraswamy generalized power Lomax distribution are derived, including moments, entropy measures and order statistics. The second approach is statistical. The maximum likelihood estimates of the parameters are described and a brief simulation study shows their effectiveness. Two real data sets are taken to show how the proposed distribution can be applied concretely; parameter estimates are obtained and fitting comparisons are performed with other well-established Lomax based distributions. The Kumaraswamy generalized power Lomax distribution turns out to be best by capturing fine details in the structure of the data considered.

Sign in / Sign up

Export Citation Format

Share Document