scholarly journals Risk sensitive control of Markov processes in countable state space

1996 ◽  
Vol 29 (3) ◽  
pp. 147-155 ◽  
Author(s):  
Daniel Hernandez-Hernández ◽  
Steven I. Marcus
Keyword(s):  
State Space ◽  
Markov Processes ◽  
Countable State Space ◽  
Risk Sensitive ◽  
Countable State ◽  
Risk Sensitive Control

Kolmogorov's differential equations for non-stationary, countable state Markov processes with uniformly continuous transition probabilities

1973 ◽  
Vol 73 (1) ◽  
pp. 119-138 ◽  
Author(s):  
Gerald S. Goodman ◽  
S. Johansen
Keyword(s):  
Markov Chain ◽  
Differential Equations ◽  
State Space ◽  
Markov Processes ◽  
Transition Probabilities ◽  
The State ◽  
Continuous Transition ◽  
Countable State Space ◽  
Countable State ◽  
Uniformly Continuous

1. SummaryWe shall consider a non-stationary Markov chain on a countable state space E. The transition probabilities {P(s, t), 0 ≤ s ≤ t <t0 ≤ ∞} are assumed to be continuous in (s, t) uniformly in the state i ε E.

THE SHANNON–MCMILLAN THEOREM FOR MARKOV CHAINS INDEXED BY A CAYLEY TREE IN RANDOM ENVIRONMENT

2017 ◽  
Vol 32 (4) ◽  
pp. 626-639 ◽  
Author(s):  
Zhiyan Shi ◽  
Pingping Zhong ◽  
Yan Fan
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Random Environment ◽  
Cayley Tree ◽  
Law Of Large Numbers ◽  
Strong Law ◽  
Countable State Space ◽  
Large Numbers ◽  
Countable State ◽  
Definition Of

In this paper, we give the definition of tree-indexed Markov chains in random environment with countable state space, and then study the realization of Markov chain indexed by a tree in random environment. Finally, we prove the strong law of large numbers and Shannon–McMillan theorem for Markov chains indexed by a Cayley tree in a Markovian environment with countable state space.

Criteria for the non-ergodicity of stochastic processes: application to the exponential back-off protocol

1987 ◽  
Vol 24 (02) ◽  
pp. 347-354 ◽  
Author(s):  
Guy Fayolle ◽  
Rudolph Iasnogorodski
Keyword(s):  
Stochastic Process ◽  
Markov Chain ◽  
Stochastic Processes ◽  
State Space ◽  
Discrete Time ◽  
Random Variables ◽  
Countable State Space ◽  
Countable State ◽  
New Criteria

In this paper, we present some simple new criteria for the non-ergodicity of a stochastic process (Yn ), n ≧ 0 in discrete time, when either the upward or downward jumps are majorized by i.i.d. random variables. This situation is encountered in many practical situations, where the (Yn ) are functionals of some Markov chain with countable state space. An application to the exponential back-off protocol is described.

Sign in / Sign up

Export Citation Format

Share Document