Zero-Sum Markov Games with Impulse Controls

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

Download Full-text

Bias and overtaking equilibria for zero-sum continuous-time Markov games

Mathematical Methods of Operations Research ◽

10.1007/s001860400392 ◽

2005 ◽

Vol 61 (3) ◽

pp. 437-454 ◽

Cited By ~ 25

Author(s):

Tomás Prieto-Rumeau ◽

Onésimo Hernández-Lerma

Keyword(s):

Continuous Time ◽

Markov Games ◽

Zero Sum

Download Full-text

Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates

Bernoulli ◽

10.3150/bj/1137421638 ◽

2005 ◽

Vol 11 (6) ◽

pp. 1009-1029 ◽

Cited By ~ 39

Author(s):

Xianping Guo ◽

Onésimo Hernández-Lerma

Keyword(s):

Continuous Time ◽

Markov Games ◽

Zero Sum

Download Full-text

Average optimal strategies for zero-sum Markov games with poorly known payoff function on one side

Journal of Dynamics & Games ◽

10.3934/jdg.2014.1.105 ◽

2014 ◽

Vol 1 (1) ◽

pp. 105-119

Author(s):

Fernando Luque-Vásquez ◽

◽

J. Adolfo Minjárez-Sosa ◽

Keyword(s):

Optimal Strategies ◽

Payoff Function ◽

Markov Games ◽

Zero Sum ◽

Average Optimal Strategies

Download Full-text

ORDERED FIELD PROPERTY IN A SUBCLASS OF FINITE SER-SIT SEMI-MARKOV GAMES

International Game Theory Review ◽

10.1142/s0219198913400264 ◽

2013 ◽

Vol 15 (04) ◽

pp. 1340026 ◽

Cited By ~ 6

Author(s):

PRASENJIT MONDAL ◽

SAGNIK SINHA

Keyword(s):

Stochastic Game ◽

Transition Probability ◽

Pollution Tax ◽

Markov Games ◽

Ordered Field ◽

Markov Game ◽

Ordered Field Property ◽

Single Matrix ◽

Zero Sum ◽

Field Property

In this paper, we deal with a subclass of two-person finite SeR-SIT (Separable Reward-State Independent Transition) semi-Markov games which can be solved by solving a single matrix/bimatrix game under discounted as well as limiting average (undiscounted) payoff criteria. A SeR-SIT semi-Markov game does not satisfy the so-called (Archimedean) ordered field property in general. Besides, the ordered field property does not hold even for a SeR-SIT-PT (Separable Reward-State-Independent Transition Probability and Time) semi-Markov game, which is a natural version of a SeR-SIT stochastic (Markov) game. However by using an additional condition, we have shown that a subclass of finite SeR-SIT-PT semi-Markov games have the ordered field property for both discounted and undiscounted semi-Markov games with both players having state-independent stationary optimals. The ordered field property also holds for the nonzero-sum case under the same assumptions. We find a relation between the values of the discounted and the undiscounted zero-sum semi-Markov games for this modified subclass. We propose a more realistic pollution tax model for this subclass of SeR-SIT semi-Markov games than pollution tax model for SeR-SIT stochastic game. Finite step algorithms are given for the discounted and for the zero-sum undiscounted cases.

Download Full-text

A Policy Improvement Algorithm for Solving a Mixture Class of Perfect Information and AR-AT Semi-Markov Games

International Game Theory Review ◽

10.1142/s0219198920400083 ◽

2020 ◽

Vol 22 (02) ◽

pp. 2040008

Author(s):

P. Mondal ◽

S. K. Neogy ◽

A. Gupta ◽

D. Ghorui

Keyword(s):

Perfect Information ◽

Policy Improvement ◽

Markov Games ◽

Markov Game ◽

Improvement Algorithm ◽

Finite State ◽

Markov Decision ◽

Mixture Class ◽

Zero Sum ◽

Action Spaces

Zero-sum two-person discounted semi-Markov games with finite state and action spaces are studied where a collection of states having Perfect Information (PI) property is mixed with another collection of states having Additive Reward–Additive Transition and Action Independent Transition Time (AR-AT-AITT) property. For such a PI/AR-AT-AITT mixture class of games, we prove the existence of an optimal pure stationary strategy for each player. We develop a policy improvement algorithm for solving discounted semi-Markov decision processes (one player version of semi-Markov games) and using it we obtain a policy-improvement type algorithm for computing an optimal strategy pair of a PI/AR-AT-AITT mixture semi-Markov game. Finally, we extend our results when the states having PI property are replaced by a subclass of Switching Control (SC) states.

Download Full-text

, a simple reinforcement learning scheme for two-player zero-sum Markov games

Neurocomputing ◽

10.1016/j.neucom.2008.12.022 ◽

2009 ◽

Vol 72 (7-9) ◽

pp. 1494-1507

Author(s):

Benoît Frénay ◽

Marco Saerens

Keyword(s):

Reinforcement Learning ◽

Markov Games ◽

Learning Scheme ◽

Zero Sum

Download Full-text