A characterization of Markov chains on infinite graphs by limiting distributions

1995 ◽  
Vol 65 (5) ◽  
pp. 449-460 ◽  
Author(s):  
Zoran Vondraček
Keyword(s):  
Markov Chains ◽  
Infinite Graphs ◽  
Limiting Distributions

Characterization of the conditional stationary distribution in Markov chains via systems of linear inequalities

2020 ◽  
Vol 52 (4) ◽  
pp. 1249-1283
Author(s):  
Masatoshi Kimura ◽  
Tetsuya Takine
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Continuous Time ◽  
State Transition ◽  
Convex Polytope ◽  
Linear Inequalities ◽  
Systems Of Linear Inequalities ◽  
Free Sets ◽  
Practical Implications

AbstractThis paper considers ergodic, continuous-time Markov chains $\{X(t)\}_{t \in (\!-\infty,\infty)}$ on $\mathbb{Z}^+=\{0,1,\ldots\}$ . For an arbitrarily fixed $N \in \mathbb{Z}^+$ , we study the conditional stationary distribution $\boldsymbol{\pi}(N)$ given the Markov chain being in $\{0,1,\ldots,N\}$ . We first characterize $\boldsymbol{\pi}(N)$ via systems of linear inequalities and identify simplices that contain $\boldsymbol{\pi}(N)$ , by examining the $(N+1) \times (N+1)$ northwest corner block of the infinitesimal generator $\textbf{\textit{Q}}$ and the subset of the first $N+1$ states whose members are directly reachable from at least one state in $\{N+1,N+2,\ldots\}$ . These results are closely related to the augmented truncation approximation (ATA), and we provide some practical implications for the ATA. Next we consider an extension of the above results, using the $(K+1) \times (K+1)$ ( $K > N$ ) northwest corner block of $\textbf{\textit{Q}}$ and the subset of the first $K+1$ states whose members are directly reachable from at least one state in $\{K+1,K+2,\ldots\}$ . Furthermore, we introduce new state transition structures called (K, N)-skip-free sets, using which we obtain the minimum convex polytope that contains $\boldsymbol{\pi}(N)$ .

Characterization of the rat exploratory behavior in the elevated plus-maze with Markov chains

2010 ◽  
Vol 193 (2) ◽  
pp. 288-295 ◽  
Author(s):  
Julián Tejada ◽  
Geraldine G. Bosco ◽  
Silvio Morato ◽  
Antonio C. Roque
Keyword(s):  
Markov Chains ◽  
Exploratory Behavior ◽  
Elevated Plus Maze ◽  
Plus Maze

Strong ergodicity for continuous-time, non-homogeneous Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 692-694 ◽  
Author(s):  
Mark Scott ◽  
Barry C. Arnold ◽  
Dean L. Isaacson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Homogeneous Markov Chain ◽  
Strong Ergodicity

Characterizations of strong ergodicity for Markov chains using mean visit times have been found by several authors (Huang and Isaacson (1977), Isaacson and Arnold (1978)). In this paper a characterization of uniform strong ergodicity for a continuous-time non-homogeneous Markov chain is given. This extends the characterization, using mean visit times, that was given by Isaacson and Arnold.

scholarly journals OBLIGATION BLACKWELL GAMES AND P-AUTOMATA

2017 ◽  
Vol 82 (2) ◽  
pp. 420-452
Author(s):  
KRISHNENDU CHATTERJEE ◽  
NIR PITERMAN
Keyword(s):  
Markov Chains ◽  
Decision Problem ◽  
Value Function ◽  
Acceptance Condition ◽  
Parity Games ◽  
The Value Function

AbstractWe generalize winning conditions in two-player games by adding a structural acceptance condition called obligations. Obligations are orthogonal to the linear winning conditions that define whether a play is winning. Obligations are a declaration that player 0 can achieve a certain value from a configuration. If the obligation is met, the value of that configuration for player 0 is 1.We define the value in such games and show that obligation games are determined. For Markov chains with Borel objectives and obligations, and finite turn-based stochastic parity games with obligations we give an alternative and simpler characterization of the value function. Based on this simpler definition we show that the decision problem of winning finite turn-based stochastic parity games with obligations is in NP∩co-NP. We also show that obligation games provide a game framework for reasoning about p-automata.

scholarly journals Subshifts on Infinite Alphabets and Their Entropy

Entropy ◽  
2020 ◽  
Vol 22 (11) ◽  
pp. 1293
Author(s):  
Sharwin Rezagholi
Keyword(s):  
Growth Rate ◽  
Markov Chains ◽  
Topological Entropy ◽  
Spectral Radius ◽  
Symbolic Dynamics ◽  
Infinite Graphs ◽  
Complexity Function ◽  
Countably Infinite

We analyze symbolic dynamics to infinite alphabets by endowing the alphabet with the cofinite topology. The topological entropy is shown to be equal to the supremum of the growth rate of the complexity function with respect to finite subalphabets. For the case of topological Markov chains induced by countably infinite graphs, our approach yields the same entropy as the approach of Gurevich We give formulae for the entropy of countable topological Markov chains in terms of the spectral radius in l2.

scholarly journals FINITE-SAMPLE SIZE CONTROL OF IVX-BASED TESTS IN PREDICTIVE REGRESSIONS

2020 ◽  
pp. 1-25
Author(s):  
Mehdi Hosseinkouchack ◽  
Matei Demetrescu
Keyword(s):  
Size Control ◽  
Finite Sample ◽  
Limiting Distributions ◽  
Valid Inference ◽  
Tuning Parameters ◽  
Predictive Regressions ◽  
Chi Squared ◽  
Standard Normal ◽  
Finite Sample Size

Abstract In predictive regressions with variables of unknown persistence, the use of extended IV (IVX) instruments leads to asymptotically valid inference. Under highly persistent regressors, the standard normal or chi-squared limiting distributions for the usual t and Wald statistics may, however, differ markedly from the actual finite-sample distributions which exhibit in particular noncentrality. Convergence to the limiting distributions is shown to occur at a rate depending on the choice of the IVX tuning parameters and can be very slow in practice. A characterization of the leading higher-order terms of the t statistic is provided for the simple regression case, which motivates finite-sample corrections. Monte Carlo simulations confirm the usefulness of the proposed methods.

On records and related processes for sequences with trends

1999 ◽  
Vol 36 (3) ◽  
pp. 668-681 ◽  
Author(s):  
K. Borovkov
Keyword(s):  
Markov Chains ◽  
Limit Theorems ◽  
Convergence Rates ◽  
Rate Function ◽  
Regular Case ◽  
Limiting Distributions ◽  
Record Times ◽  
Ergodicity Theorem ◽  
Linear Trends

We study the records and related variables for sequences with linear trends. We discuss the properties of the asymptotic rate function and relationships between the distribution of the long-term maxima in the sequence and that of a particular observation, including two characterization type results. We also consider certain Markov chains related to the process of records and prove limit theorems for them, including the ergodicity theorem in the regular case (convergence rates are given under additional assumptions), and derive the limiting distributions for the inter-record times and increments of records.

scholarly journals Limiting distributions for countable state topological Markov chains with holes

2017 ◽  
Vol 37 (1) ◽  
pp. 105-130 ◽  
Author(s):  
Mark F. Demers ◽  
◽  
Christopher J. Ianzano ◽  
Philip Mayer ◽  
Peter Morfe ◽  
...  
Keyword(s):  
Markov Chains ◽  
Limiting Distributions ◽  
Countable State

scholarly journals Characterization of dry period in the transfrontier Bia watershed in Côte d’Ivoire and Ghana: contribution of Markov chains

2015 ◽  
Vol 24 (3) ◽  
pp. 186-197
Author(s):  
N’Diaye Hermann Meledje ◽  
Kouakou Lazare Kouassi ◽  
Yao Alexis N’Go ◽  
Issiaka Savane
Keyword(s):  
Markov Chains ◽  
Cote D'ivoire ◽  
Côte D’Ivoire ◽  
Dry Period ◽  
Cote D’Ivoire ◽  
Côte D'ivoire
Sign in / Sign up

Export Citation Format

Share Document