Perfect information two-person zero-sum markov games with imprecise transition probabilities

Zero-sum two-person discounted semi-Markov games with finite state and action spaces are studied where a collection of states having Perfect Information (PI) property is mixed with another collection of states having Additive Reward–Additive Transition and Action Independent Transition Time (AR-AT-AITT) property. For such a PI/AR-AT-AITT mixture class of games, we prove the existence of an optimal pure stationary strategy for each player. We develop a policy improvement algorithm for solving discounted semi-Markov decision processes (one player version of semi-Markov games) and using it we obtain a policy-improvement type algorithm for computing an optimal strategy pair of a PI/AR-AT-AITT mixture semi-Markov game. Finally, we extend our results when the states having PI property are replaced by a subclass of Switching Control (SC) states.

Download Full-text

Linear Programming and Zero-Sum Two-Person Undiscounted Semi-Markov Games

Asia Pacific Journal of Operational Research ◽

10.1142/s0217595915500438 ◽

2015 ◽

Vol 32 (06) ◽

pp. 1550043 ◽

Cited By ~ 6

Author(s):

Prasenjit Mondal

Keyword(s):

Linear Programming ◽

Transition Probabilities ◽

Optimal Solution ◽

Programming Algorithm ◽

Stationary Strategy ◽

Markov Games ◽

Markov Decision ◽

Transition Times ◽

Optimal Stationary Strategies ◽

Zero Sum

In this paper, zero-sum two-person finite undiscounted (limiting average) semi-Markov games (SMGs) are considered. We prove that the solutions of the game when both players are restricted to semi-Markov strategies are solutions for the original game. In addition, we show that if one player fixes a stationary strategy, then the other player can restrict himself in solving an undiscounted semi-Markov decision process associated with that stationary strategy. The undiscounted SMGs are also studied when the transition probabilities and the transition times are controlled by a fixed player in all states. If such games are unichain, we prove that the value and optimal stationary strategies of the players can be obtained from an optimal solution of a linear programming algorithm. We propose a realistic and generalized traveling inspection model that suitably fits into the class of one player control undiscounted unichain semi-Markov games.

Download Full-text

Converging Coevolutionary Algorithm for Two-Person Zero-Sum Discounted Markov Games With Perfect Information

IEEE Transactions on Automatic Control ◽

10.1109/tac.2007.914299 ◽

2008 ◽

Vol 53 (2) ◽

pp. 596-601 ◽

Cited By ~ 4

Author(s):

Hyeong Soo Chang

Keyword(s):

Perfect Information ◽

Markov Games ◽

Coevolutionary Algorithm ◽

Zero Sum

Download Full-text

Ordered Field Property for Semi-Markov Games when One Player Controls Transition Probabilities and Transition Times

International Game Theory Review ◽

10.1142/s0219198915400228 ◽

2015 ◽

Vol 17 (02) ◽

pp. 1540022 ◽

Cited By ~ 9

Author(s):

Prasenjit Mondal ◽

Sagnik Sinha

Keyword(s):

Linear Programming ◽

Transition Probabilities ◽

Markov Games ◽

Markov Strategy ◽

Ordered Field ◽

Player Game ◽

Ordered Field Property ◽

Transition Times ◽

Zero Sum ◽

Field Property

Two-person finite semi-Markov games (SMGs) are studied when the transition probabilities and the transition times are controlled by one player at all states. For the discounted games in this class, we prove that the ordered field property holds and there exist optimal/Nash equilibrium stationary strategies for the players. We illustrate that the zero-sum SMGs where only transition probabilities are controlled by one player, do not necessarily satisfy the ordered field property. An algorithm along with a numerical example for the discounted one player control zero-sum SMGs is given via linear programming. For the undiscounted version of such games, we exhibit with an example that if the game ceases to be unichain, an optimal stationary or Markov strategy need not exist, (though in this example of a one-player game we exhibit a semi-stationary optimal strategy/policy). Lastly, we prove that if such games are unichain, then they possess the ordered field property for the undiscounted case as well.

Download Full-text