Zero-sum two-person semi-Markov games

Semi-Markov games are investigated under discounted and limiting average payoff criteria. The issue of the existence of the value and a pair of stationary optimal strategies are settled; the optimality equation is studied and under a natural ergodic condition the existence of a solution to the optimality equation is proved for the limiting average case. Semi-Markov games provide useful flexibility in constructing recursive game models. All the work on Markov/semi-Markov decision processes and Markov (stochastic) games can be viewed as special cases of the developments in this paper.

Download Full-text

PIVOTING ALGORITHMS FOR SOME CLASSES OF STOCHASTIC GAMES: A SURVEY

International Game Theory Review ◽

10.1142/s0219198901000385 ◽

2001 ◽

Vol 03 (02n03) ◽

pp. 253-281 ◽

Cited By ~ 21

Author(s):

S. R. MOHAN ◽

S. K. NEOGY ◽

T. PARTHASARATHY

Keyword(s):

Linear Complementarity Problem ◽

Complementarity Problem ◽

Stochastic Games ◽

Optimal Strategies ◽

Linear Complementarity ◽

Dual Simplex Method ◽

Special Cases ◽

Dual Simplex ◽

Finite Step ◽

Zero Sum

In this paper, we survey the recent literature on computing the value vector and the associated optimal strategies of the players for special cases of zero-sum stochastic games, or in computing a Nash equilibrium point and the corresponding stationary strategies of the players for special cases of nonzero-sum stochastic games, using finite-step algorithms based on pivoting. Examples of finite-step pivoting algorithms are the various simplex-type algorithms, such as the primal simplex or dual simplex method for solving the linear programming problem or Lemke's or Lemke-Howson's algorithm for solving the linear complementarity problem. Also included are Lemke-type algorithms for solving various generalisations of the linear complementarity problem. The survey also includes a few new results and observations.

Download Full-text

On the optimality equation for zero-sum ergodic stochastic games

Mathematical Methods of Operations Research ◽

10.1007/s001860100144 ◽

2001 ◽

Vol 54 (2) ◽

pp. 291-301 ◽

Cited By ~ 15

Author(s):

Anna Jaśkiewicz ◽

Andrzej S. Nowak

Keyword(s):

Stochastic Games ◽

Optimality Equation ◽

Zero Sum

Download Full-text

On zero-sum two-person undiscounted semi-Markov games with a multichain structure

Advances in Applied Probability ◽

10.1017/apr.2017.23 ◽

2017 ◽

Vol 49 (3) ◽

pp. 826-849 ◽

Cited By ~ 2

Author(s):

Prasenjit Mondal

Keyword(s):

Stochastic Game ◽

Optimal Strategies ◽

Markov Games ◽

Stationary Strategies ◽

Current State ◽

Strategy Evaluation ◽

Optimality Equations ◽

Optimal Stationary Strategies ◽

Zero Sum ◽

Absorbing States

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

Download Full-text

Average optimal strategies for zero-sum Markov games with poorly known payoff function on one side

Journal of Dynamics & Games ◽

10.3934/jdg.2014.1.105 ◽

2014 ◽

Vol 1 (1) ◽

pp. 105-119

Author(s):

Fernando Luque-Vásquez ◽

◽

J. Adolfo Minjárez-Sosa ◽

Keyword(s):

Optimal Strategies ◽

Payoff Function ◽

Markov Games ◽

Zero Sum ◽

Average Optimal Strategies

Download Full-text

A Policy Improvement Algorithm for Solving a Mixture Class of Perfect Information and AR-AT Semi-Markov Games

International Game Theory Review ◽

10.1142/s0219198920400083 ◽

2020 ◽

Vol 22 (02) ◽

pp. 2040008

Author(s):

P. Mondal ◽

S. K. Neogy ◽

A. Gupta ◽

D. Ghorui

Keyword(s):

Perfect Information ◽

Policy Improvement ◽

Markov Games ◽

Markov Game ◽

Improvement Algorithm ◽

Finite State ◽

Markov Decision ◽

Mixture Class ◽

Zero Sum ◽

Action Spaces

Zero-sum two-person discounted semi-Markov games with finite state and action spaces are studied where a collection of states having Perfect Information (PI) property is mixed with another collection of states having Additive Reward–Additive Transition and Action Independent Transition Time (AR-AT-AITT) property. For such a PI/AR-AT-AITT mixture class of games, we prove the existence of an optimal pure stationary strategy for each player. We develop a policy improvement algorithm for solving discounted semi-Markov decision processes (one player version of semi-Markov games) and using it we obtain a policy-improvement type algorithm for computing an optimal strategy pair of a PI/AR-AT-AITT mixture semi-Markov game. Finally, we extend our results when the states having PI property are replaced by a subclass of Switching Control (SC) states.

Download Full-text

Asymptotically Optimal Strategies for Adaptive Zero-Sum Discounted Markov Games

SIAM Journal on Control and Optimization ◽

10.1137/060651458 ◽

2009 ◽

Vol 48 (3) ◽

pp. 1405-1421 ◽

Cited By ~ 6

Author(s):

J. Adolfo Minjárez-Sosa ◽

Oscar Vega-Amaya

Keyword(s):

Optimal Strategies ◽

Asymptotically Optimal ◽

Markov Games ◽

Zero Sum

Download Full-text

Linear Programming and Zero-Sum Two-Person Undiscounted Semi-Markov Games

Asia Pacific Journal of Operational Research ◽

10.1142/s0217595915500438 ◽

2015 ◽

Vol 32 (06) ◽

pp. 1550043 ◽

Cited By ~ 6

Author(s):

Prasenjit Mondal

Keyword(s):

Linear Programming ◽

Transition Probabilities ◽

Optimal Solution ◽

Programming Algorithm ◽

Stationary Strategy ◽

Markov Games ◽

Markov Decision ◽

Transition Times ◽

Optimal Stationary Strategies ◽

Zero Sum

In this paper, zero-sum two-person finite undiscounted (limiting average) semi-Markov games (SMGs) are considered. We prove that the solutions of the game when both players are restricted to semi-Markov strategies are solutions for the original game. In addition, we show that if one player fixes a stationary strategy, then the other player can restrict himself in solving an undiscounted semi-Markov decision process associated with that stationary strategy. The undiscounted SMGs are also studied when the transition probabilities and the transition times are controlled by a fixed player in all states. If such games are unichain, we prove that the value and optimal stationary strategies of the players can be obtained from an optimal solution of a linear programming algorithm. We propose a realistic and generalized traveling inspection model that suitably fits into the class of one player control undiscounted unichain semi-Markov games.

Download Full-text