scholarly journals Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

2007 ◽  
Vol 39 (03) ◽  
pp. 645-668 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Processes ◽  
Continuous Time ◽  
Infinitesimal Operator ◽  
Polish Spaces ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Value Of The Game ◽  
Optimal Stationary Strategies ◽  
Zero Sum ◽  
Jump Markov Processes

This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to beunbounded, and the payoff rates may haveneither upper nor lower bounds. We give conditions on the game'sprimitive dataunder which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using theextended infinitesimal operatorassociated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide arecursiveway of computing (or at least approximating) the value of the game. Moreover, we present a ‘martingale characterization’ of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.

scholarly journals Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

2007 ◽  
Vol 39 (3) ◽  
pp. 645-668 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Processes ◽  
Continuous Time ◽  
Infinitesimal Operator ◽  
Polish Spaces ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Value Of The Game ◽  
Optimal Stationary Strategies ◽  
Zero Sum ◽  
Jump Markov Processes

This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. We give conditions on the game's primitive data under which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using the extended infinitesimal operator associated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide a recursive way of computing (or at least approximating) the value of the game. Moreover, we present a ‘martingale characterization’ of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

2003 ◽  
Vol 40 (02) ◽  
pp. 327-345 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Rates ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Continuous Time Markov Chains ◽  
Average Payoff ◽  
Optimal Stationary Strategies ◽  
Zero Sum

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

2003 ◽  
Vol 40 (2) ◽  
pp. 327-345 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Rates ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Continuous Time Markov Chains ◽  
Average Payoff ◽  
Optimal Stationary Strategies ◽  
Zero Sum

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.

On zero-sum two-person undiscounted semi-Markov games with a multichain structure

2017 ◽  
Vol 49 (3) ◽  
pp. 826-849 ◽  
Author(s):  
Prasenjit Mondal
Keyword(s):  
Stochastic Game ◽  
Optimal Strategies ◽  
Markov Games ◽  
Stationary Strategies ◽  
Current State ◽  
Strategy Evaluation ◽  
Optimality Equations ◽  
Optimal Stationary Strategies ◽  
Zero Sum ◽  
Absorbing States

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

Sign in / Sign up

Export Citation Format

Share Document