A characterization of potential kernels for recurrent Markov chains with strong Feller transition function

1973 ◽  
pp. 213-238 ◽  
Author(s):  
Ryōji Kondō ◽  
Yōichi Ōshima
Keyword(s):  
Markov Chains ◽  
Transition Function ◽  
Strong Feller

Characterization of the conditional stationary distribution in Markov chains via systems of linear inequalities

2020 ◽  
Vol 52 (4) ◽  
pp. 1249-1283
Author(s):  
Masatoshi Kimura ◽  
Tetsuya Takine
Keyword(s):  
Markov Chains ◽  
Stationary Distribution ◽  
Continuous Time ◽  
State Transition ◽  
Convex Polytope ◽  
Linear Inequalities ◽  
Systems Of Linear Inequalities ◽  
Free Sets ◽  
Practical Implications

AbstractThis paper considers ergodic, continuous-time Markov chains $\{X(t)\}_{t \in (\!-\infty,\infty)}$ on $\mathbb{Z}^+=\{0,1,\ldots\}$ . For an arbitrarily fixed $N \in \mathbb{Z}^+$ , we study the conditional stationary distribution $\boldsymbol{\pi}(N)$ given the Markov chain being in $\{0,1,\ldots,N\}$ . We first characterize $\boldsymbol{\pi}(N)$ via systems of linear inequalities and identify simplices that contain $\boldsymbol{\pi}(N)$ , by examining the $(N+1) \times (N+1)$ northwest corner block of the infinitesimal generator $\textbf{\textit{Q}}$ and the subset of the first $N+1$ states whose members are directly reachable from at least one state in $\{N+1,N+2,\ldots\}$ . These results are closely related to the augmented truncation approximation (ATA), and we provide some practical implications for the ATA. Next we consider an extension of the above results, using the $(K+1) \times (K+1)$ ( $K > N$ ) northwest corner block of $\textbf{\textit{Q}}$ and the subset of the first $K+1$ states whose members are directly reachable from at least one state in $\{K+1,K+2,\ldots\}$ . Furthermore, we introduce new state transition structures called (K, N)-skip-free sets, using which we obtain the minimum convex polytope that contains $\boldsymbol{\pi}(N)$ .

Characterization of the rat exploratory behavior in the elevated plus-maze with Markov chains

2010 ◽  
Vol 193 (2) ◽  
pp. 288-295 ◽  
Author(s):  
Julián Tejada ◽  
Geraldine G. Bosco ◽  
Silvio Morato ◽  
Antonio C. Roque
Keyword(s):  
Markov Chains ◽  
Exploratory Behavior ◽  
Elevated Plus Maze ◽  
Plus Maze

Strong ergodicity for continuous-time, non-homogeneous Markov chains

1982 ◽  
Vol 19 (3) ◽  
pp. 692-694 ◽  
Author(s):  
Mark Scott ◽  
Barry C. Arnold ◽  
Dean L. Isaacson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Homogeneous Markov Chain ◽  
Strong Ergodicity

Characterizations of strong ergodicity for Markov chains using mean visit times have been found by several authors (Huang and Isaacson (1977), Isaacson and Arnold (1978)). In this paper a characterization of uniform strong ergodicity for a continuous-time non-homogeneous Markov chain is given. This extends the characterization, using mean visit times, that was given by Isaacson and Arnold.

scholarly journals OBLIGATION BLACKWELL GAMES AND P-AUTOMATA

2017 ◽  
Vol 82 (2) ◽  
pp. 420-452
Author(s):  
KRISHNENDU CHATTERJEE ◽  
NIR PITERMAN
Keyword(s):  
Markov Chains ◽  
Decision Problem ◽  
Value Function ◽  
Acceptance Condition ◽  
Parity Games ◽  
The Value Function

AbstractWe generalize winning conditions in two-player games by adding a structural acceptance condition called obligations. Obligations are orthogonal to the linear winning conditions that define whether a play is winning. Obligations are a declaration that player 0 can achieve a certain value from a configuration. If the obligation is met, the value of that configuration for player 0 is 1.We define the value in such games and show that obligation games are determined. For Markov chains with Borel objectives and obligations, and finite turn-based stochastic parity games with obligations we give an alternative and simpler characterization of the value function. Based on this simpler definition we show that the decision problem of winning finite turn-based stochastic parity games with obligations is in NP∩co-NP. We also show that obligation games provide a game framework for reasoning about p-automata.

scholarly journals Characterization of dry period in the transfrontier Bia watershed in Côte d’Ivoire and Ghana: contribution of Markov chains

2015 ◽  
Vol 24 (3) ◽  
pp. 186-197
Author(s):  
N’Diaye Hermann Meledje ◽  
Kouakou Lazare Kouassi ◽  
Yao Alexis N’Go ◽  
Issiaka Savane
Keyword(s):  
Markov Chains ◽  
Cote D'ivoire ◽  
Côte D’Ivoire ◽  
Dry Period ◽  
Cote D’Ivoire ◽  
Côte D'ivoire

Algebraic characterization of infinite Markov chains where movement to the right is limited to one step

1977 ◽  
Vol 14 (04) ◽  
pp. 740-747 ◽  
Author(s):  
Ester Samuel-Cahn ◽  
Shmuel Zamir
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Necessary Conditions ◽  
Sufficient Conditions ◽  
Algebraic Characterization ◽  
Transient States ◽  
One Step ◽  
The Right

We consider an infinite Markov chain with states E 0, E 1, …, such that E 1, E 2, … is not closed, and for i ≧ 1 movement to the right is limited by one step. Simple algebraic characterizations are given for persistency of all states, and, if E 0 is absorbing, simple expressions are given for the probabilities of staying forever among the transient states. Examples are furnished, and simple necessary conditions and sufficient conditions for the above characterizations are given.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

2003 ◽  
Vol 40 (02) ◽  
pp. 327-345 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Rates ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Continuous Time Markov Chains ◽  
Average Payoff ◽  
Optimal Stationary Strategies ◽  
Zero Sum

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.

Sign in / Sign up

Export Citation Format

Share Document