Quotients of Markov chains and asymptotic properties of the stationary distribution of the Markov chain associated to an evolutionary algorithm

2007 ◽  
Vol 9 (2) ◽  
pp. 109-123 ◽  
Author(s):  
Boris Mitavskiy ◽  
Jonathan E. Rowe ◽  
Alden Wright ◽  
Lothar M. Schmitt
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Evolutionary Algorithm ◽  
Stationary Distribution ◽  
Asymptotic Properties

Estimating the Ratios of the Stationary Distribution Values for Markov Chains Modeling Evolutionary Algorithms

2009 ◽  
Vol 17 (3) ◽  
pp. 343-377 ◽  
Author(s):  
Boris Mitavskiy ◽  
Chris Cannings
Keyword(s):  
Stochastic Process ◽  
Population Genetics ◽  
Markov Chain ◽  
Evolutionary Algorithms ◽  
Markov Chains ◽  
Evolutionary Algorithm ◽  
Stationary Distribution ◽  
Genetic Load ◽  
Construction Method ◽  
Expected Time

The evolutionary algorithm stochastic process is well-known to be Markovian. These have been under investigation in much of the theoretical evolutionary computing research. When the mutation rate is positive, the Markov chain modeling of an evolutionary algorithm is irreducible and, therefore, has a unique stationary distribution. Rather little is known about the stationary distribution. In fact, the only quantitative facts established so far tell us that the stationary distributions of Markov chains modeling evolutionary algorithms concentrate on uniform populations (i.e., those populations consisting of a repeated copy of the same individual). At the same time, knowing the stationary distribution may provide some information about the expected time it takes for the algorithm to reach a certain solution, assessment of the biases due to recombination and selection, and is of importance in population genetics to assess what is called a “genetic load” (see the introduction for more details). In the recent joint works of the first author, some bounds have been established on the rates at which the stationary distribution concentrates on the uniform populations. The primary tool used in these papers is the “quotient construction” method. It turns out that the quotient construction method can be exploited to derive much more informative bounds on ratios of the stationary distribution values of various subsets of the state space. In fact, some of the bounds obtained in the current work are expressed in terms of the parameters involved in all the three main stages of an evolutionary algorithm: namely, selection, recombination, and mutation.

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (2) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

Markov Chains in Many Dimensions

1994 ◽  
Vol 26 (3) ◽  
pp. 756-774 ◽  
Author(s):  
Dimitris N. Politis
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Maximum Entropy ◽  
Stationary Distribution ◽  
Random Fields ◽  
Special Class ◽  
Markov Random Fields ◽  
One Dimension ◽  
Stationary Markov Chain ◽  
Markov Random

A generalization of the notion of a stationary Markov chain in more than one dimension is proposed, and is found to be a special class of homogeneous Markov random fields. Stationary Markov chains in many dimensions are shown to possess a maximum entropy property, analogous to the corresponding property for Markov chains in one dimension. In addition, a representation of Markov chains in many dimensions is provided, together with a method for their generation that converges to their stationary distribution.

The existence of moments for stationary Markov chains

1983 ◽  
Vol 20 (01) ◽  
pp. 191-196 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Time Dependent ◽  
General Function ◽  
Convergence Of Moments ◽  
Special Cases

We give conditions under which the stationary distribution π of a Markov chain admits moments of the general form ∫ f(x)π(dx), where f is a general function; specific examples include f(x) = xr and f(x) = esx . In general the time-dependent moments of the chain then converge to the stationary moments. We show that in special cases this convergence of moments occurs at a geometric rate. The results are applied to random walk on [0, ∞).

scholarly journals Normal distributions of finite Markov chains

2019 ◽  
Vol 29 (08) ◽  
pp. 1431-1449
Author(s):  
John Rhodes ◽  
Anne Schilling
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Planar Graphs ◽  
Finite Semigroup ◽  
Finite Markov Chain ◽  
Normal Distributions ◽  
Straight Line ◽  
Finite Markov Chains ◽  
The Right

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The loops touch only at one vertex either of the straight line or of another attached loop. Our analysis is based on our previous work, which derives the stationary distribution of a finite Markov chain using semaphore codes on the Karnofsky–Rhodes and McCammond expansion of the right Cayley graph of the finite semigroup underlying the Markov chain.

On the existence of quasi-stationary distributions in denumerable R-transient Markov chains

1992 ◽  
Vol 29 (01) ◽  
pp. 21-36 ◽  
Author(s):  
Masaaki Kijima
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
State Space ◽  
Stationary Distribution ◽  
Practical Importance ◽  
Prime Concern ◽  
Continuous Random Walk ◽  
Transient Markov Chain ◽  
Quasi Stationary Distribution

Let {Xn, n= 0, 1, 2, ···} be a transient Markov chain which, when restricted to the state space 𝒩+= {1, 2, ···}, is governed by an irreducible, aperiodic and strictly substochastic matrix𝐏= (pij), and letpij(n) =P∈Xn=j, Xk∈ 𝒩+fork= 0, 1, ···,n|X0=i],i, j𝒩+. The prime concern of this paper is conditions for the existence of the limits,qijsay, ofasn →∞. Ifthe distribution (qij) is called the quasi-stationary distribution of {Xn} and has considerable practical importance. It will be shown that, under some conditions, if a non-negative non-trivial vectorx= (xi) satisfyingrxT=xT𝐏andexists, whereris the convergence norm of𝐏, i.e.r=R–1andand T denotes transpose, then it is unique, positive elementwise, andqij(n) necessarily converge toxjasn →∞.Unlike existing results in the literature, our results can be applied even to theR-null andR-transient cases. Finally, an application to a left-continuous random walk whose governing substochastic matrix isR-transient is discussed to demonstrate the usefulness of our results.

scholarly journals SIMPLE PROCEDURES FOR FINDING MEAN FIRST PASSAGE TIMES IN MARKOV CHAINS

2007 ◽  
Vol 24 (06) ◽  
pp. 813-829 ◽  
Author(s):  
JEFFREY J. HUNTER
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Linear Equations ◽  
First Passage Times ◽  
First Passage ◽  
Mean First Passage Times ◽  
The Mean ◽  
Computational Procedures ◽  
Passage Times

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell's fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse of I - P + eπT where eT = (1, 1, …, 1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.

scholarly journals SERIES EXPANSIONS FOR FINITE-STATE MARKOV CHAINS

2007 ◽  
Vol 21 (3) ◽  
pp. 381-400 ◽  
Author(s):  
Bernd Heidergott ◽  
Arie Hordijk ◽  
Miranda van Uitert
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Numerical Algorithm ◽  
Series Expansions ◽  
Finite Markov Chain ◽  
Numerical Examples ◽  
Finite State

This article provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

Perturbation theory and finite Markov chains

1968 ◽  
Vol 5 (02) ◽  
pp. 401-413 ◽  
Author(s):  
Paul J. Schweitzer
Keyword(s):  
Perturbation Theory ◽  
Markov Chain ◽  
Markov Chains ◽  
Stationary Distribution ◽  
Fundamental Matrix ◽  
Transition Probabilities ◽  
Semi Group ◽  
Partial Derivatives ◽  
Finite Markov Chains ◽  
Derivatives Of

A perturbation formalism is presented which shows how the stationary distribution and fundamental matrix of a Markov chain containing a single irreducible set of states change as the transition probabilities vary. Expressions are given for the partial derivatives of the stationary distribution and fundamental matrix with respect to the transition probabilities. Semi-group properties of the generators of transformations from one Markov chain to another are investigated. It is shown that a perturbation formalism exists in the multiple subchain case if and only if the change in the transition probabilities does not alter the number of, or intermix the various subchains. The formalism is presented when this condition is satisfied.

Sign in / Sign up

Export Citation Format

Share Document