On Simulating a Markov Chain Stationary Distribution when Transition Probabilities are Unknown

1995 ◽  
pp. 1-9 ◽  
Author(s):  
David Aldous
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Transition Probabilities

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (2) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

A Markov chain associated with the minimal quasi-stationary distribution of birth–death chains

1995 ◽  
Vol 32 (01) ◽  
pp. 25-38
Author(s):  
Servet Martínez ◽  
Maria Eulália Vares
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Conditional Distribution ◽  
Transition Probabilities ◽  
The Matrix ◽  
Quasi Stationary Distribution ◽  
Birth Death

We show that if the limiting conditional distribution for an absorbed birth–death chain exists, then the chain conditioned to non-absorption converges to a Markov chain with transition probabilities given by the matrix associated with the minimal quasi-stationary distribution.

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (02) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

A Markov chain associated with the minimal quasi-stationary distribution of birth–death chains

1995 ◽  
Vol 32 (1) ◽  
pp. 25-38 ◽  
Author(s):  
Servet Martínez ◽  
Maria Eulália Vares
Keyword(s):  
Markov Chain ◽  
Stationary Distribution ◽  
Conditional Distribution ◽  
Transition Probabilities ◽  
The Matrix ◽  
Quasi Stationary Distribution ◽  
Birth Death

We show that if the limiting conditional distribution for an absorbed birth–death chain exists, then the chain conditioned to non-absorption converges to a Markov chain with transition probabilities given by the matrix associated with the minimal quasi-stationary distribution.

Data Mining Technique Applied To DNA Sequencing

2015 ◽  
Vol 2 (7) ◽  
pp. 18
Author(s):  
R. Jamuna
Keyword(s):  
Data Mining ◽  
Dna Methylation ◽  
Markov Chain ◽  
Human Genome ◽  
Transition Probabilities ◽  
Markov Chain Model ◽  
Effective Means ◽  
Cpg Islands ◽  
Chain Model ◽  
The Given

CpG islands (CGIs) play a vital role in genome analysis as genomic markers.  Identification of the CpG pair has contributed not only to the prediction of promoters but also to the understanding of the epigenetic causes of cancer. In the human genome [1] wherever the dinucleotides CG occurs the C nucleotide (cytosine) undergoes chemical modifications. There is a relatively high probability of this modification that mutates C into a T. For biologically important reasons the mutation modification process is suppressed in short stretches of the genome, such as ‘start’ regions. In these regions [2] predominant CpG dinucleotides are found than elsewhere. Such regions are called CpG islands. DNA methylation is an effective means by which gene expression is silenced. In normal cells, DNA methylation functions to prevent the expression of imprinted and inactive X chromosome genes. In cancerous cells, DNA methylation inactivates tumor-suppressor genes, as well as DNA repair genes, can disrupt cell-cycle regulation. The most current methods for identifying CGIs suffered from various limitations and involved a lot of human interventions. This paper gives an easy searching technique with data mining of Markov Chain in genes. Markov chain model has been applied to study the probability of occurrence of C-G pair in the given   gene sequence. Maximum Likelihood estimators for the transition probabilities for each model and analgously for the  model has been developed and log odds ratio that is calculated estimates the presence or absence of CpG is lands in the given gene which brings in many  facts for the cancer detection in human genome.

scholarly journals Calibration of Transition Intensities for a Multistate Model: Application to Long-Term Care

Risks ◽  
2021 ◽  
Vol 9 (2) ◽  
pp. 37
Author(s):  
Manuel L. Esquível ◽  
Gracinda R. Guerreiro ◽  
Matilde C. Oliveira ◽  
Pedro Corte Real
Keyword(s):  
Markov Chain ◽  
Continuous Time ◽  
Long Term Care ◽  
Transition Probabilities ◽  
Markov Chain Model ◽  
Chain Model ◽  
Chain Process ◽  
Term Care ◽  
National Network

We consider a non-homogeneous continuous time Markov chain model for Long-Term Care with five states: the autonomous state, three dependent states of light, moderate and severe dependence levels and the death state. For a general approach, we allow for non null intensities for all the returns from higher dependence levels to all lesser dependencies in the multi-state model. Using data from the 2015 Portuguese National Network of Continuous Care database, as the main research contribution of this paper, we propose a method to calibrate transition intensities with the one step transition probabilities estimated from data. This allows us to use non-homogeneous continuous time Markov chains for modeling Long-Term Care. We solve numerically the Kolmogorov forward differential equations in order to obtain continuous time transition probabilities. We assess the quality of the calibration using the Portuguese life expectancies. Based on reasonable monthly costs for each dependence state we compute, by Monte Carlo simulation, trajectories of the Markov chain process and derive relevant information for model validation and premium calculation.

Approximating the stationary distribution of an infinite stochastic matrix

1991 ◽  
Vol 28 (1) ◽  
pp. 96-103 ◽  
Author(s):  
Daniel P. Heyman
Keyword(s):  
Markov Chain ◽  
Steady State ◽  
Stationary Distribution ◽  
Numerical Approximation ◽  
Stochastic Matrix ◽  
Sufficient Conditions ◽  
Finite System ◽  
Balance Equations ◽  
The Given ◽  
Steady State Balance

We are given a Markov chain with states 0, 1, 2, ···. We want to get a numerical approximation of the steady-state balance equations. To do this, we truncate the chain, keeping the first n states, make the resulting matrix stochastic in some convenient way, and solve the finite system. The purpose of this paper is to provide some sufficient conditions that imply that as n tends to infinity, the stationary distributions of the truncated chains converge to the stationary distribution of the given chain. Our approach is completely probabilistic, and our conditions are given in probabilistic terms. We illustrate how to verify these conditions with five examples.

scholarly journals Stationary distribution Markov chain for Covid-19 pandemic

2021 ◽  
Vol 1722 ◽  
pp. 012084
Author(s):  
A L H Achmad ◽  
Mahrudinda ◽  
B N Ruchjana
Keyword(s):  
Markov Chain ◽  
Stationary Distribution

scholarly journals A Bayesian model for binary Markov chains

2004 ◽  
Vol 2004 (8) ◽  
pp. 421-429 ◽  
Author(s):  
Souad Assoudou ◽  
Belkheir Essebbar
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chains ◽  
Bayesian Estimation ◽  
Bayesian Model ◽  
Transition Probabilities ◽  
Simulated Data ◽  
Bayesian Estimator ◽  
Jeffreys Prior ◽  
Data Set

This note is concerned with Bayesian estimation of the transition probabilities of a binary Markov chain observed from heterogeneous individuals. The model is founded on the Jeffreys' prior which allows for transition probabilities to be correlated. The Bayesian estimator is approximated by means of Monte Carlo Markov chain (MCMC) techniques. The performance of the Bayesian estimates is illustrated by analyzing a small simulated data set.

Sign in / Sign up

Export Citation Format

Share Document