ORDERED FIELD PROPERTY IN A SUBCLASS OF FINITE SER-SIT SEMI-MARKOV GAMES

In this paper, we deal with a subclass of two-person finite SeR-SIT (Separable Reward-State Independent Transition) semi-Markov games which can be solved by solving a single matrix/bimatrix game under discounted as well as limiting average (undiscounted) payoff criteria. A SeR-SIT semi-Markov game does not satisfy the so-called (Archimedean) ordered field property in general. Besides, the ordered field property does not hold even for a SeR-SIT-PT (Separable Reward-State-Independent Transition Probability and Time) semi-Markov game, which is a natural version of a SeR-SIT stochastic (Markov) game. However by using an additional condition, we have shown that a subclass of finite SeR-SIT-PT semi-Markov games have the ordered field property for both discounted and undiscounted semi-Markov games with both players having state-independent stationary optimals. The ordered field property also holds for the nonzero-sum case under the same assumptions. We find a relation between the values of the discounted and the undiscounted zero-sum semi-Markov games for this modified subclass. We propose a more realistic pollution tax model for this subclass of SeR-SIT semi-Markov games than pollution tax model for SeR-SIT stochastic game. Finite step algorithms are given for the discounted and for the zero-sum undiscounted cases.

Download Full-text

Ordered Field Property for Semi-Markov Games when One Player Controls Transition Probabilities and Transition Times

International Game Theory Review ◽

10.1142/s0219198915400228 ◽

2015 ◽

Vol 17 (02) ◽

pp. 1540022 ◽

Cited By ~ 9

Author(s):

Prasenjit Mondal ◽

Sagnik Sinha

Keyword(s):

Linear Programming ◽

Transition Probabilities ◽

Markov Games ◽

Markov Strategy ◽

Ordered Field ◽

Player Game ◽

Ordered Field Property ◽

Transition Times ◽

Zero Sum ◽

Field Property

Two-person finite semi-Markov games (SMGs) are studied when the transition probabilities and the transition times are controlled by one player at all states. For the discounted games in this class, we prove that the ordered field property holds and there exist optimal/Nash equilibrium stationary strategies for the players. We illustrate that the zero-sum SMGs where only transition probabilities are controlled by one player, do not necessarily satisfy the ordered field property. An algorithm along with a numerical example for the discounted one player control zero-sum SMGs is given via linear programming. For the undiscounted version of such games, we exhibit with an example that if the game ceases to be unichain, an optimal stationary or Markov strategy need not exist, (though in this example of a one-player game we exhibit a semi-stationary optimal strategy/policy). Lastly, we prove that if such games are unichain, then they possess the ordered field property for the undiscounted case as well.

Download Full-text

On a Mixture Class of Stochastic Game with Ordered Field Property

Mathematical Programming and Game Theory for Decision Making - Statistical Science and Interdisciplinary Research ◽

10.1142/9789812813220_0025 ◽

2008 ◽

pp. 451-477 ◽

Cited By ~ 3

Author(s):

S. K. Neogy ◽

A. K. Das ◽

S. Sinha ◽

A. Gupta

Keyword(s):

Stochastic Game ◽

Ordered Field ◽

Mixture Class ◽

Ordered Field Property ◽

Field Property

Download Full-text

On zero-sum two-person undiscounted semi-Markov games with a multichain structure

Advances in Applied Probability ◽

10.1017/apr.2017.23 ◽

2017 ◽

Vol 49 (3) ◽

pp. 826-849 ◽

Cited By ~ 2

Author(s):

Prasenjit Mondal

Keyword(s):

Stochastic Game ◽

Optimal Strategies ◽

Markov Games ◽

Stationary Strategies ◽

Current State ◽

Strategy Evaluation ◽

Optimality Equations ◽

Optimal Stationary Strategies ◽

Zero Sum ◽

Absorbing States

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

Download Full-text

DISCOUNTING AND AVERAGING IN GAMES ACROSS TIME SCALES

International Journal of Foundations of Computer Science ◽

10.1142/s0129054112400308 ◽

2012 ◽

Vol 23 (03) ◽

pp. 609-625 ◽

Cited By ~ 1

Author(s):

KRISHNENDU CHATTERJEE ◽

RUPAK MAJUMDAR

Keyword(s):

Time Scales ◽

Stochastic Game ◽

Optimal Strategies ◽

Sequential Decision ◽

Ordered Field ◽

Markov Decision ◽

Mean Payoff Games ◽

Upper Level ◽

Ordered Field Property ◽

Mean Payoff

We introduce two-level discounted and mean-payoff games played by two players on a perfect-information stochastic game graph. The upper level game is a discounted or mean-payoff game and the lower level game is a (undiscounted) reachability game. Two-level games model hierarchical and sequential decision making under uncertainty across different time scales. For both discounted and mean-payoff two-level games, we show the existence of pure memoryless optimal strategies for both players and an ordered field property. We show that if there is only one player (Markov decision processes), then the values can be computed in polynomial time. It follows that whether the value of a player is equal to a given rational constant in two-level discounted or mean-payoff games can be decided in NP ∩ coNP . We also give an alternate strategy improvement algorithm to compute the value.

Download Full-text

A Policy Improvement Algorithm for Solving a Mixture Class of Perfect Information and AR-AT Semi-Markov Games

International Game Theory Review ◽

10.1142/s0219198920400083 ◽

2020 ◽

Vol 22 (02) ◽

pp. 2040008

Author(s):

P. Mondal ◽

S. K. Neogy ◽

A. Gupta ◽

D. Ghorui

Keyword(s):

Perfect Information ◽

Policy Improvement ◽

Markov Games ◽

Markov Game ◽

Improvement Algorithm ◽

Finite State ◽

Markov Decision ◽

Mixture Class ◽

Zero Sum ◽

Action Spaces

Zero-sum two-person discounted semi-Markov games with finite state and action spaces are studied where a collection of states having Perfect Information (PI) property is mixed with another collection of states having Additive Reward–Additive Transition and Action Independent Transition Time (AR-AT-AITT) property. For such a PI/AR-AT-AITT mixture class of games, we prove the existence of an optimal pure stationary strategy for each player. We develop a policy improvement algorithm for solving discounted semi-Markov decision processes (one player version of semi-Markov games) and using it we obtain a policy-improvement type algorithm for computing an optimal strategy pair of a PI/AR-AT-AITT mixture semi-Markov game. Finally, we extend our results when the states having PI property are replaced by a subclass of Switching Control (SC) states.

Download Full-text