scholarly journals L-cumulants, L-cumulant embeddings and algebraic statis- tics

2012 ◽  
Vol 3 (1) ◽  
Author(s):  
Piotr Zwiernik
Keyword(s):  
Markov Processes ◽  
Statistical Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Random Variables ◽  
Algebraic Varieties ◽  
Main Motivation ◽  
Hidden Markov Processes ◽  
Probabilistic Setting ◽  
Tree Models

Focusing on the discrete probabilistic setting we generalize the combinatorial denitionof cumulants to L-cumulants. This generalization keeps all the desired properties of the classicalcumulants like semi-invariance and vanishing for independent blocks of random variables. Theseproperties make L-cumulants useful for the algebraic analysis of statistical models. We illustratethis for general Markov models and hidden Markov processes in the case when the hidden processis binary. The main motivation of this work is to understand cumulant-like coordinates in algebraicstatistics and to give a more insightful explanation why tree cumulants give such an elegantdescription of binary hidden tree models. Moreover, we argue that L-cumulants can be used in theanalysis of certain classical algebraic varieties.

Hidden Markov Processes

2014 ◽  
Author(s):  
M. Vidyasagar
Keyword(s):  
Hidden Markov Models ◽  
Markov Processes ◽  
Viterbi Algorithm ◽  
Markov Models ◽  
Hidden Markov ◽  
Local Alignment ◽  
Biological Applications ◽  
Standard Material ◽  
Hidden Markov Processes ◽  
Genomics And Proteomics

This book explores important aspects of Markov and hidden Markov processes and the applications of these ideas to various problems in computational biology. It starts from first principles, so that no previous knowledge of probability is necessary. However, the work is rigorous and mathematical, making it useful to engineers and mathematicians, even those not interested in biological applications. A range of exercises is provided, including drills to familiarize the reader with concepts and more advanced problems that require deep thinking about the theory. Biological applications are taken from post-genomic biology, especially genomics and proteomics. The topics examined include standard material such as the Perron–Frobenius theorem, transient and recurrent states, hitting probabilities and hitting times, maximum likelihood estimation, the Viterbi algorithm, and the Baum–Welch algorithm. The book contains discussions of extremely useful topics not usually seen at the basic level, such as ergodicity of Markov processes, Markov Chain Monte Carlo (MCMC), information theory, and large deviation theory for both i.i.d and Markov processes. It also presents state-of-the-art realization theory for hidden Markov models. Among biological applications, it offers an in-depth look at the BLAST (Basic Local Alignment Search Technique) algorithm, including a comprehensive explanation of the underlying theory. Other applications such as profile hidden Markov models are also explored.

scholarly journals Shannon Entropy Rate of Hidden Markov Processes

2021 ◽  
Vol 183 (2) ◽  
Author(s):  
Alexandra M. Jurgens ◽  
James P. Crutchfield
Keyword(s):  
Markov Processes ◽  
Shannon Entropy ◽  
Statistical Models ◽  
Hidden Markov ◽  
Entropy Rate ◽  
Minimal Set ◽  
Fundamental Physics ◽  
Hidden Markov Processes ◽  
Finite State ◽  
Finite Expression

AbstractHidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously complicated, however, even if the chain is finite state: no finite expression for their Shannon entropy rate exists, as the set of their predictive features is generically infinite. As such, to date one cannot make general statements about how random they are nor how structured. Here, we address the first part of this challenge by showing how to efficiently and accurately calculate their entropy rates. We also show how this method gives the minimal set of infinite predictive features. A sequel addresses the challenge’s second part on structure.

Generic Identication of Binary-Valued Hidden Markov Processes

2014 ◽  
Vol 5 (1) ◽  
Author(s):  
Alexander Schönhuth
Keyword(s):  
Stochastic Process ◽  
Markov Process ◽  
General Solution ◽  
Parameter Space ◽  
Markov Processes ◽  
Hidden Markov ◽  
Probability Distributions ◽  
Algebraic Statistics ◽  
Hidden Markov Processes ◽  
Lower Dimensional

The generic identication problem is to decide whether a stochastic process (Xt) is ahidden Markov process and if yes to infer its parameters for all but a subset of parametrizationsthat form a lower-dimensional subvariety in parameter space. Partial answers so far availabledepend on extra assumptions on the processes, which are usually centered around stationarity.Here we present a general solution for binary-valued hidden Markov processes. Our approach isrooted in algebraic statistics hence it is geometric in nature. We nd that the algebraic varietiesassociated with the probability distributions of binary-valued hidden Markov processes are zerosets of determinantal equations which draws a connection to well-studied objects from algebra. Asa consequence, our solution allows for algorithmic implementation based on elementary (linear)algebraic routines.

scholarly journals Analysis of the plant architecture via tree-structured statistical models: the hidden Markov tree models

2005 ◽  
Vol 166 (3) ◽  
pp. 813-825 ◽  
Author(s):  
J.-B. Durand ◽  
Y. Guédon ◽  
Y. Caraglio ◽  
E. Costes
Keyword(s):  
Statistical Models ◽  
Plant Architecture ◽  
Hidden Markov ◽  
Markov Tree ◽  
Tree Models

Norm-Observable Operator Models

2010 ◽  
Vol 22 (7) ◽  
pp. 1927-1959 ◽  
Author(s):  
Ming-Jie Zhao ◽  
Herbert Jaeger
Keyword(s):  
Time Series ◽  
Markov Chain ◽  
Statistical Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Linear Operators ◽  
Negative Probability ◽  
Novel Variant ◽  
System States ◽  
Mathematical Foundations

Hidden Markov models (HMMs) are one of the most popular and successful statistical models for time series. Observable operator models (OOMs) are generalizations of HMMs that exhibit several attractive advantages. In particular, a variety of highly efficient, constructive, and asymptotically correct learning algorithms are available for OOMs. However, the OOM theory suffers from the negative probability problem (NPP): a given, learned OOM may sometimes predict negative probabilities for certain events. It was recently shown that it is undecidable whether a given OOM will eventually produce such negative values. We propose a novel variant of OOMs, called norm-observable operator models (NOOMs), which avoid the NPP by design. Like OOMs, NOOMs use a set of linear operators to update system states. But differing from OOMs, they represent probabilities by the square of the norm of system states, thus precluding negative probability values. While being free of the NPP, NOOMs retain most advantages of OOMs. For example, NOOMs also capture (some) processes that cannot be modeled by HMMs. More importantly, in principle, NOOMs can be learned from data in a constructive way, and the learned models are asymptotically correct. We also prove that NOOMs capture all Markov chain (MC) describable processes. This letter presents the mathematical foundations of NOOMs, discusses the expressiveness of the model class, and explains how a NOOM can be estimated from data constructively.

Sign in / Sign up

Export Citation Format

Share Document