scholarly journals Minimum Message Length Inference of the Exponential Distribution with Type I Censoring

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1439
Author(s):  
Enes Makalic ◽  
Daniel Francis Schmidt
Keyword(s):  
Maximum Likelihood ◽  
Exponential Distribution ◽  
Statistical Models ◽  
Selection Criterion ◽  
Information Criterion ◽  
Type I ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Length Estimate

Data with censoring is common in many areas of science and the associated statistical models are generally estimated with the method of maximum likelihood combined with a model selection criterion such as Akaike’s information criterion. This manuscript demonstrates how the information theoretic minimum message length principle can be used to estimate statistical models in the presence of type I random and fixed censoring data. The exponential distribution with fixed and random censoring is used as an example to demonstrate the process where we observe that the minimum message length estimate of mean survival time has some advantages over the standard maximum likelihood estimate.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

Entropy ◽  
2021 ◽  
Vol 23 (12) ◽  
pp. 1601
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Environmental Science ◽  
Short Term Memory ◽  
Arima Model ◽  
Theoretic Approach ◽  
Arma Models ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Real World Datasets

Modeling and analysis of time series are important in applications including economics, engineering, environmental science and social science. Selecting the best time series model with accurate parameters in forecasting is a challenging objective for scientists and academic researchers. Hybrid models combining neural networks and traditional Autoregressive Moving Average (ARMA) models are being used to improve the accuracy of modeling and forecasting time series. Most of the existing time series models are selected by information-theoretic approaches, such as AIC, BIC, and HQ. This paper revisits a model selection technique based on Minimum Message Length (MML) and investigates its use in hybrid time series analysis. MML is a Bayesian information-theoretic approach and has been used in selecting the best ARMA model. We utilize the long short-term memory (LSTM) approach to construct a hybrid ARMA-LSTM model and show that MML performs better than AIC, BIC, and HQ in selecting the model—both in the traditional ARMA models (without LSTM) and with hybrid ARMA-LSTM models. These results held on simulated data and both real-world datasets that we considered. We also develop a simple MML ARIMA model.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting

2021 ◽  
Author(s):  
Zheng Fang ◽  
David L. Dowe ◽  
Shelton Peiris ◽  
Dedi Rosadi
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
Arma Model ◽  
Autoregressive Moving Average ◽  
Information Theoretic ◽  
Minimum Message Length ◽  
Message Length ◽  
Long Short Term Memory

We investigate the power of time series analysis based on a variety of information-theoretic approaches from statistics (AIC, BIC) and machine learning (Minimum Message Length) - and we then compare their efficacy with traditional time series model and with hybrids involving deep learning. More specifically, we develop AIC, BIC and Minimum Message Length (MML) ARMA (autoregressive moving average) time series models - with this Bayesian information-theoretic MML ARMA modelling already being new work. We then study deep learning based algorithms in time series forecasting, using Long Short Term Memory (LSTM), and we then combine this with the ARMA modelling to produce a hybrid ARMA-LSTM prediction. Part of the purpose of the use of LSTM is to seek capture any hidden information in the residuals left from the traditional ARMA model. We show that MML not only outperforms earlier statistical approaches to ARMA modelling, but we further show that the hybrid MML ARMA-LSTM models outperform both ARMA models and LSTM models.

scholarly journals Left‑Censored Samples from Skewed Distributions: Statistical Inference and Applications

2018 ◽  
Vol 66 (1) ◽  
pp. 245-252
Author(s):  
Michal Fusek ◽  
Jaroslav Michálek
Keyword(s):  
Maximum Likelihood ◽  
Statistical Inference ◽  
Exponential Distribution ◽  
Information Matrix ◽  
Computational Procedure ◽  
Maximum Likelihood Estimates ◽  
Likelihood Method ◽  
Type I ◽  
Censored Samples ◽  
Asymptotic Tests

Left‑censored data occur frequently in many areas. At present, researchers pay attention to skewed censored distributions more frequently. This paper deals with statistical inference of type I multiply left‑censored Weibull and exponential distributions. It suggests a computational procedure for calculation of maximum likelihood estimates of the parameters. The expected Fisher information matrix for estimation of variances of estimated parameters is introduced. The estimates are then used for construction of confidence intervals for the expectation using the maximum likelihood method. Asymptotic tests for comparison of distributions (expectations respectively) of two independent left‑censored Weibull samples are proposed. Furthermore, asymptotic tests for assessing suitability of reduction of the Weibull distribution to the exponential distribution are introduced. Finally, the left‑censored exponential distribution is briefly described. Methods derived in this paper are illustrated on elemental carbon measurements, and can be applied in analysis of real environmental and/or chemical data.

Large-sample confidence intervals of information-theoretic measures in linguistics

2020 ◽  
Vol 6 (1) ◽  
pp. 19-54
Author(s):  
Ryan Ka Yau Lai ◽  
Youngah Do
Keyword(s):  
Maximum Likelihood ◽  
Corpus Linguistics ◽  
Delta Method ◽  
Confidence Bounds ◽  
Likelihood Estimator ◽  
Information Theoretic ◽  
Leibler Divergence ◽  
Information Theoretic Measures ◽  
Data Points ◽  
Measure Of Uncertainty

This article explores a method of creating confidence bounds for information-theoretic measures in linguistics, such as entropy, Kullback-Leibler Divergence (KLD), and mutual information. We show that a useful measure of uncertainty can be derived from simple statistical principles, namely the asymptotic distribution of the maximum likelihood estimator (MLE) and the delta method. Three case studies from phonology and corpus linguistics are used to demonstrate how to apply it and examine its robustness against common violations of its assumptions in linguistics, such as insufficient sample size and non-independence of data points.

The Comparison Between the Bayes Estimator and the Maximum Likelihood Estimator of the Reliability Function for Negative Exponential Distribution

2018 ◽  
pp. 439
Author(s):  
Hazim Mansour Gorgees ◽  
Bushra Abdualrasool Ali ◽  
Raghad Ibrahim Kathum
Keyword(s):  
Maximum Likelihood ◽  
Maximum Likelihood Estimator ◽  
Exponential Distribution ◽  
Reliability Function ◽  
Bayes Estimator ◽  
Likelihood Estimator ◽  
Monte Carlo Simulation Technique ◽  
Negative Exponential Distribution ◽  
Negative Exponential ◽  
Better Than

     In this paper, the maximum likelihood estimator and the Bayes estimator of the reliability function for negative exponential distribution has been derived, then a Monte –Carlo simulation technique was employed to compare the performance of such estimators. The integral mean square error (IMSE) was used as a criterion for this comparison. The simulation results displayed that the Bayes estimator performed better than the maximum likelihood estimator for different samples sizes.

scholarly journals Maximum Likelihood Analysis of Rare Binary Traits Under Different Modes of Inheritance

Genetics ◽  
1996 ◽  
Vol 143 (4) ◽  
pp. 1819-1829 ◽  
Author(s):  
G Thaller ◽  
L Dempfle ◽  
I Hoeschele
Keyword(s):  
Maximum Likelihood ◽  
Mixed Model ◽  
Major Gene ◽  
Likelihood Ratio Statistic ◽  
Information Criterion ◽  
Genetic Models ◽  
Parameter Estimates ◽  
Gene Effects ◽  
Binary Traits ◽  
Deterministic Algorithms

Abstract Maximum likelihood methodology was applied to determine the mode of inheritance of rare binary traits with data structures typical for swine populations. The genetic models considered included a monogenic, a digenic, a polygenic, and three mixed polygenic and major gene models. The main emphasis was on the detection of major genes acting on a polygenic background. Deterministic algorithms were employed to integrate and maximize likelihoods. A simulation study was conducted to evaluate model selection and parameter estimation. Three designs were simulated that differed in the number of sires/number of dams within sires (10/10, 30/30, 100/30). Major gene effects of at least one SD of the liability were detected with satisfactory power under the mixed model of inheritance, except for the smallest design. Parameter estimates were empirically unbiased with acceptable standard errors, except for the smallest design, and allowed to distinguish clearly between the genetic models. Distributions of the likelihood ratio statistic were evaluated empirically, because asymptotic theory did not hold. For each simulation model, the Average Information Criterion was computed for all models of analysis. The model with the smallest value was chosen as the best model and was equal to the true model in almost every case studied.

Sign in / Sign up

Export Citation Format

Share Document