On zero-sum two-person undiscounted semi-Markov games with a multichain structure

2017 ◽  
Vol 49 (3) ◽  
pp. 826-849 ◽  
Author(s):  
Prasenjit Mondal
Keyword(s):  
Stochastic Game ◽  
Optimal Strategies ◽  
Markov Games ◽  
Stationary Strategies ◽  
Current State ◽  
Strategy Evaluation ◽  
Optimality Equations ◽  
Optimal Stationary Strategies ◽  
Zero Sum ◽  
Absorbing States

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

scholarly journals On the existence of stationary Nash equilibria in average stochastic games with finite state and action spaces

2020 ◽  
Vol 13 ◽  
pp. 304-323
Author(s):  
Dmitrii Lozovanu ◽  
◽  
Stefan Pickl ◽  
Keyword(s):  
Nash Equilibria ◽  
Stochastic Games ◽  
Optimal Strategies ◽  
Stationary Strategies ◽  
Finite State ◽  
Optimal Stationary Strategies ◽  
Action Spaces

We consider infinite n-person stochastic games with limiting average payoffs criteria for the players. The main results of the paper are concerned with the existence of stationary Nash equilibria and determining the optimal strategies of the players in the games with finite state and action spaces. We present conditions for the existence of stationary Nash equilibria in the considered games and propose an approach for determining the optimal stationary strategies of the players if such strategies exist.

scholarly journals Average optimal strategies for zero-sum Markov games with poorly known payoff function on one side

2014 ◽  
Vol 1 (1) ◽  
pp. 105-119
Author(s):  
Fernando Luque-Vásquez ◽  
◽  
J. Adolfo Minjárez-Sosa ◽  
Keyword(s):  
Optimal Strategies ◽  
Payoff Function ◽  
Markov Games ◽  
Zero Sum ◽  
Average Optimal Strategies

ORDERED FIELD PROPERTY IN A SUBCLASS OF FINITE SER-SIT SEMI-MARKOV GAMES

2013 ◽  
Vol 15 (04) ◽  
pp. 1340026 ◽  
Author(s):  
PRASENJIT MONDAL ◽  
SAGNIK SINHA
Keyword(s):  
Stochastic Game ◽  
Transition Probability ◽  
Pollution Tax ◽  
Markov Games ◽  
Ordered Field ◽  
Markov Game ◽  
Ordered Field Property ◽  
Single Matrix ◽  
Zero Sum ◽  
Field Property

In this paper, we deal with a subclass of two-person finite SeR-SIT (Separable Reward-State Independent Transition) semi-Markov games which can be solved by solving a single matrix/bimatrix game under discounted as well as limiting average (undiscounted) payoff criteria. A SeR-SIT semi-Markov game does not satisfy the so-called (Archimedean) ordered field property in general. Besides, the ordered field property does not hold even for a SeR-SIT-PT (Separable Reward-State-Independent Transition Probability and Time) semi-Markov game, which is a natural version of a SeR-SIT stochastic (Markov) game. However by using an additional condition, we have shown that a subclass of finite SeR-SIT-PT semi-Markov games have the ordered field property for both discounted and undiscounted semi-Markov games with both players having state-independent stationary optimals. The ordered field property also holds for the nonzero-sum case under the same assumptions. We find a relation between the values of the discounted and the undiscounted zero-sum semi-Markov games for this modified subclass. We propose a more realistic pollution tax model for this subclass of SeR-SIT semi-Markov games than pollution tax model for SeR-SIT stochastic game. Finite step algorithms are given for the discounted and for the zero-sum undiscounted cases.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

2003 ◽  
Vol 40 (02) ◽  
pp. 327-345 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Rates ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Continuous Time Markov Chains ◽  
Average Payoff ◽  
Optimal Stationary Strategies ◽  
Zero Sum

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.

Asymptotically Optimal Strategies for Adaptive Zero-Sum Discounted Markov Games

2009 ◽  
Vol 48 (3) ◽  
pp. 1405-1421 ◽  
Author(s):  
J. Adolfo Minjárez-Sosa ◽  
Oscar Vega-Amaya
Keyword(s):  
Optimal Strategies ◽  
Asymptotically Optimal ◽  
Markov Games ◽  
Zero Sum

Linear Programming and Zero-Sum Two-Person Undiscounted Semi-Markov Games

2015 ◽  
Vol 32 (06) ◽  
pp. 1550043 ◽  
Author(s):  
Prasenjit Mondal
Keyword(s):  
Linear Programming ◽  
Transition Probabilities ◽  
Optimal Solution ◽  
Programming Algorithm ◽  
Stationary Strategy ◽  
Markov Games ◽  
Markov Decision ◽  
Transition Times ◽  
Optimal Stationary Strategies ◽  
Zero Sum

In this paper, zero-sum two-person finite undiscounted (limiting average) semi-Markov games (SMGs) are considered. We prove that the solutions of the game when both players are restricted to semi-Markov strategies are solutions for the original game. In addition, we show that if one player fixes a stationary strategy, then the other player can restrict himself in solving an undiscounted semi-Markov decision process associated with that stationary strategy. The undiscounted SMGs are also studied when the transition probabilities and the transition times are controlled by a fixed player in all states. If such games are unichain, we prove that the value and optimal stationary strategies of the players can be obtained from an optimal solution of a linear programming algorithm. We propose a realistic and generalized traveling inspection model that suitably fits into the class of one player control undiscounted unichain semi-Markov games.

scholarly journals On the Existence and Determining Stationary Nash Equilibria for Switching Controller Stochastic Games

2021 ◽  
Vol 14 ◽  
pp. 290-301
Author(s):  
Dmitrii Lozovanu ◽  
◽  
Stefan Pickl ◽  
Keyword(s):  
Nash Equilibrium ◽  
Normal Form ◽  
Nash Equilibria ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Static Game ◽  
Stationary Strategies ◽  
Switching Controller ◽  
Optimal Stationary Strategies ◽  
Stationary Nash Equilibrium

In this paper we consider the problem of the existence and determining stationary Nash equilibria for switching controller stochastic games with discounted and average payoffs. The set of states and the set of actions in the considered games are assumed to be finite. For a switching controller stochastic game with discounted payoffs we show that all stationary equilibria can be found by using an auxiliary continuous noncooperative static game in normal form in which the payoffs are quasi-monotonic (quasi-convex and quasi-concave) with respect to the corresponding strategies of the players. Based on this we propose an approach for determining the optimal stationary strategies of the players. In the case of average payoffs for a switching controller stochastic game we also formulate an auxiliary noncooperative static game in normal form with quasi-monotonic payoffs and show that such a game possesses a Nash equilibrium if the corresponding switching controller stochastic game has a stationary Nash equilibrium.

scholarly journals Optimal strategies for adaptive zero-sum average Markov games

2013 ◽  
Vol 402 (1) ◽  
pp. 44-56 ◽  
Author(s):  
J. Adolfo Minjárez-Sosa ◽  
Óscar Vega-Amaya
Keyword(s):  
Optimal Strategies ◽  
Markov Games ◽  
Zero Sum

scholarly journals Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs

2007 ◽  
Vol 39 (03) ◽  
pp. 645-668 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Processes ◽  
Continuous Time ◽  
Infinitesimal Operator ◽  
Polish Spaces ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Value Of The Game ◽  
Optimal Stationary Strategies ◽  
Zero Sum ◽  
Jump Markov Processes

This paper is devoted to the study of two-person zero-sum games for continuous-time jump Markov processes with a discounted payoff criterion. The state and action spaces are all Polish spaces, the transition rates are allowed to beunbounded, and the payoff rates may haveneither upper nor lower bounds. We give conditions on the game'sprimitive dataunder which the existence of a solution to the Shapley equation is ensured. Then, from the Shapley equation, we obtain the existence of the value of the game and of a pair of optimal stationary strategies using theextended infinitesimal operatorassociated with the transition function of a possibly nonhomogeneous continuous-time jump Markov process. We also provide arecursiveway of computing (or at least approximating) the value of the game. Moreover, we present a ‘martingale characterization’ of a pair of optimal stationary strategies. Finally, we apply our results to a controlled birth and death system and a Schlögl first model, and then we use controlled Potlach processes to illustrate our conditions.

Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates

2003 ◽  
Vol 40 (2) ◽  
pp. 327-345 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Transition Rates ◽  
Zero Sum Games ◽  
Stationary Strategies ◽  
Continuous Time Markov Chains ◽  
Average Payoff ◽  
Optimal Stationary Strategies ◽  
Zero Sum

This paper is a first study of two-person zero-sum games for denumerable continuous-time Markov chains determined by given transition rates, with an average payoff criterion. The transition rates are allowed to be unbounded, and the payoff rates may have neither upper nor lower bounds. In the spirit of the ‘drift and monotonicity’ conditions for continuous-time Markov processes, we give conditions on the controlled system's primitive data under which the existence of the value of the game and a pair of strong optimal stationary strategies is ensured by using the Shapley equations. Also, we present a ‘martingale characterization’ of a pair of strong optimal stationary strategies. Our results are illustrated with a birth-and-death game.

Sign in / Sign up

Export Citation Format

Share Document