Existence of $p$-equilibrium and optimal stationary strategies in stochastic games

We consider infinite n-person stochastic games with limiting average payoffs criteria for the players. The main results of the paper are concerned with the existence of stationary Nash equilibria and determining the optimal strategies of the players in the games with finite state and action spaces. We present conditions for the existence of stationary Nash equilibria in the considered games and propose an approach for determining the optimal stationary strategies of the players if such strategies exist.

Download Full-text

On the Existence and Determining Stationary Nash Equilibria for Switching Controller Stochastic Games

Contributions to Game Theory and Management ◽

10.21638/11701/spbu31.2021.21 ◽

2021 ◽

Vol 14 ◽

pp. 290-301

Author(s):

Dmitrii Lozovanu ◽

◽

Stefan Pickl ◽

Keyword(s):

Nash Equilibrium ◽

Normal Form ◽

Nash Equilibria ◽

Stochastic Games ◽

Stochastic Game ◽

Static Game ◽

Stationary Strategies ◽

Switching Controller ◽

Optimal Stationary Strategies ◽

Stationary Nash Equilibrium

In this paper we consider the problem of the existence and determining stationary Nash equilibria for switching controller stochastic games with discounted and average payoffs. The set of states and the set of actions in the considered games are assumed to be finite. For a switching controller stochastic game with discounted payoffs we show that all stationary equilibria can be found by using an auxiliary continuous noncooperative static game in normal form in which the payoffs are quasi-monotonic (quasi-convex and quasi-concave) with respect to the corresponding strategies of the players. Based on this we propose an approach for determining the optimal stationary strategies of the players. In the case of average payoffs for a switching controller stochastic game we also formulate an auxiliary noncooperative static game in normal form with quasi-monotonic payoffs and show that such a game possesses a Nash equilibrium if the corresponding switching controller stochastic game has a stationary Nash equilibrium.

Download Full-text

Optimal stationary strategies in leavable Markov decision processes

Journal of Applied Probability ◽

10.1017/s0021900200038481 ◽

1990 ◽

Vol 27 (01) ◽

pp. 134-145

Author(s):

Matthias Fassbender

Keyword(s):

Transition Probabilities ◽

Practical Applications ◽

Stationary Strategies ◽

Countable State ◽

Markov Decision ◽

Bias Optimality ◽

The Mean ◽

Optimal Stationary Strategies ◽

The Cost ◽

Reward Criterion

This paper establishes the existence of an optimal stationary strategy in a leavable Markov decision process with countable state space and undiscounted total reward criterion. Besides assumptions of boundedness and continuity, an assumption is imposed on the model which demands the continuity of the mean recurrence times on a subset of the stationary strategies, the so-called ‘good strategies'. For practical applications it is important that this assumption is implied by an assumption about the cost structure and the transition probabilities. In the last part we point out that our results in general cannot be deduced from related works on bias-optimality by Dekker and Hordijk, Wijngaard or Mann.

Download Full-text

A Differentiable Path-Following Method to Compute Subgame Perfect Equilibria in Stationary Strategies in Robust Stochastic Games and Its Applications.

European Journal of Operational Research ◽

10.1016/j.ejor.2021.06.059 ◽

2021 ◽

Author(s):

Yiyin Cao ◽

Chuangyin Dang ◽

Zhongdong Xiao

Keyword(s):

Stochastic Games ◽

Path Following ◽

Stationary Strategies ◽

Subgame Perfect Equilibria ◽

Perfect Equilibria

Download Full-text

Optimal stationary strategies in leavable Markov decision processes

Journal of Applied Probability ◽

10.2307/3214601 ◽

1990 ◽

Vol 27 (1) ◽

pp. 134-145

Author(s):

Matthias Fassbender

Keyword(s):

Transition Probabilities ◽

Practical Applications ◽

Stationary Strategies ◽

Countable State ◽

Markov Decision ◽

Bias Optimality ◽

The Mean ◽

Optimal Stationary Strategies ◽

The Cost ◽

Reward Criterion

This paper establishes the existence of an optimal stationary strategy in a leavable Markov decision process with countable state space and undiscounted total reward criterion.Besides assumptions of boundedness and continuity, an assumption is imposed on the model which demands the continuity of the mean recurrence times on a subset of the stationary strategies, the so-called ‘good strategies'. For practical applications it is important that this assumption is implied by an assumption about the cost structure and the transition probabilities. In the last part we point out that our results in general cannot be deduced from related works on bias-optimality by Dekker and Hordijk, Wijngaard or Mann.

Download Full-text

STATIONARY STRATEGIES IN ZERO-SUM STOCHASTIC GAMES

International Game Theory Review ◽

10.1142/s0219198901000464 ◽

2001 ◽

Vol 03 (04) ◽

pp. 283-290

Author(s):

J. FLESCH ◽

F. THUIJSMAN ◽

O. J. VRIEZE

Keyword(s):

Finite Number ◽

Stochastic Games ◽

Stationary Strategies ◽

Pure Strategies ◽

Zero Sum

We deal with zero-sum stochastic games. We demonstrate the importance of stationary strategies by showing that stationary strategies are better (in terms of the rewards they guarantee for a player, against any strategy of his opponent) than (1) pure strategies (even history-dependent ones), (2) strategies which may use only a finite number of different mixed actions in any state, and (3) strategies with finite recall. Examples are given to clarify the issues.

Download Full-text

On zero-sum two-person undiscounted semi-Markov games with a multichain structure

Advances in Applied Probability ◽

10.1017/apr.2017.23 ◽

2017 ◽

Vol 49 (3) ◽

pp. 826-849 ◽

Cited By ~ 2

Author(s):

Prasenjit Mondal

Keyword(s):

Stochastic Game ◽

Optimal Strategies ◽

Markov Games ◽

Stationary Strategies ◽

Current State ◽

Strategy Evaluation ◽

Optimality Equations ◽

Optimal Stationary Strategies ◽

Zero Sum ◽

Absorbing States

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

Download Full-text