scholarly journals On the Existence and Determining Stationary Nash Equilibria for Switching Controller Stochastic Games

2021 ◽  
Vol 14 ◽  
pp. 290-301
Author(s):  
Dmitrii Lozovanu ◽  
◽  
Stefan Pickl ◽  
Keyword(s):  
Nash Equilibrium ◽  
Normal Form ◽  
Nash Equilibria ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Static Game ◽  
Stationary Strategies ◽  
Switching Controller ◽  
Optimal Stationary Strategies ◽  
Stationary Nash Equilibrium

In this paper we consider the problem of the existence and determining stationary Nash equilibria for switching controller stochastic games with discounted and average payoffs. The set of states and the set of actions in the considered games are assumed to be finite. For a switching controller stochastic game with discounted payoffs we show that all stationary equilibria can be found by using an auxiliary continuous noncooperative static game in normal form in which the payoffs are quasi-monotonic (quasi-convex and quasi-concave) with respect to the corresponding strategies of the players. Based on this we propose an approach for determining the optimal stationary strategies of the players. In the case of average payoffs for a switching controller stochastic game we also formulate an auxiliary noncooperative static game in normal form with quasi-monotonic payoffs and show that such a game possesses a Nash equilibrium if the corresponding switching controller stochastic game has a stationary Nash equilibrium.

A Characterization of Stationary Nash Equilibria of Single Controller Constrained Stochastic Games

2015 ◽  
Vol 17 (02) ◽  
pp. 1540018
Author(s):  
Vikas Vikram Singh ◽  
N. Hemachandra
Keyword(s):  
Nash Equilibrium ◽  
Nash Equilibria ◽  
Mathematical Program ◽  
Transition Probabilities ◽  
Stochastic Game ◽  
Quadratic Program ◽  
Linear Programs ◽  
Average Case ◽  
Single Controller ◽  
Stationary Nash Equilibrium

We consider a two player finite state-action general sum single controller constrained stochastic game with both discounted and average cost criteria. We consider the situation where player 1 has subscription-based constraints and player 2, who controls the transition probabilities, has realization-based constraints which can also depend on the strategies of player 1. It is known that a stationary Nash equilibrium for discounted case exists under strong Slater condition, while, for the average case, stationary Nash equilibrium exists if additionally the Markov chain is unichain. For each case we show that the set of stationary Nash equilibria of this game has one to one correspondence with the set of global minimizers of a certain nonconvex mathematical program. If the constraints of player 2 do not depend on the strategies of player 1, then the mathematical program reduces to a quadratic program. The known linear programs for zero sum games of this class can be obtained as a special case of above quadratic programs.

scholarly journals On the existence of stationary Nash equilibria in average stochastic games with finite state and action spaces

2020 ◽  
Vol 13 ◽  
pp. 304-323
Author(s):  
Dmitrii Lozovanu ◽  
◽  
Stefan Pickl ◽  
Keyword(s):  
Nash Equilibria ◽  
Stochastic Games ◽  
Optimal Strategies ◽  
Stationary Strategies ◽  
Finite State ◽  
Optimal Stationary Strategies ◽  
Action Spaces

We consider infinite n-person stochastic games with limiting average payoffs criteria for the players. The main results of the paper are concerned with the existence of stationary Nash equilibria and determining the optimal strategies of the players in the games with finite state and action spaces. We present conditions for the existence of stationary Nash equilibria in the considered games and propose an approach for determining the optimal stationary strategies of the players if such strategies exist.

BLACKWELL OPTIMALITY IN STOCHASTIC GAMES

2013 ◽  
Vol 15 (04) ◽  
pp. 1340025
Author(s):  
VIKAS VIKRAM SINGH ◽  
N. HEMACHANDRA ◽  
K. S. MALLIKARJUNA RAO
Keyword(s):  
Nash Equilibrium ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Discount Factor ◽  
State Action ◽  
Blackwell Optimality ◽  
Finite State ◽  
Markov Decision ◽  
Stationary Nash Equilibrium ◽  
General Stochastic

Blackwell optimality in a finite state-action discounted Markov decision process (MDP) gives an optimal strategy which is optimal for every discount factor close enough to one. In this article we explore this property, which we call as Blackwell–Nash equilibrium, in two player finite state-action discounted stochastic games. A strategy pair is said to be a Blackwell–Nash equilibrium if it is a Nash equilibrium for every discount factor close enough to one. A stationary Blackwell–Nash equilibrium in a stochastic game may not always exist as can be seen from "Big Match" example where a stationary Nash equilibrium does not exist in undiscounted case. For a Single Controller Additive Reward (SC-AR) stochastic game, we show that there exists a stationary deterministic Blackwell–Nash equilibrium which is also a Nash equilibrium for undiscounted case. For general stochastic games, we give some conditions which together are sufficient for any stationary Nash equilibrium of a discounted stochastic game to be a Blackwell–Nash equilibrium and it is also a Nash equilibrium of an undiscounted stochastic game. We illustrate our results on general stochastic games through a variant of the pollution tax model.

Quantifying Commitment in Nash Equilibria

2019 ◽  
Vol 21 (02) ◽  
pp. 1940011
Author(s):  
Thomas A. Weber
Keyword(s):  
Nash Equilibrium ◽  
Normal Form ◽  
Nash Equilibria ◽  
Dynamic Game ◽  
Subgame Perfect Equilibrium ◽  
Canonical Extension ◽  
Perfect Equilibrium ◽  
Form Game ◽  
Normal Form Game ◽  
The Given

To quantify a player’s commitment in a given Nash equilibrium of a finite dynamic game, we map the corresponding normal-form game to a “canonical extension,” which allows each player to adjust his or her move with a certain probability. The commitment measure relates to the average overall adjustment probabilities for which the given Nash equilibrium can be implemented as a subgame-perfect equilibrium in the canonical extension.

Existence of p-Equilibrium and Optimal Stationary Strategies in Stochastic Games

1976 ◽  
Vol 60 (1) ◽  
pp. 245 ◽  
Author(s):  
C. J. Himmelberg ◽  
T. Parthasarathy ◽  
T. E. S. Raghavan ◽  
F. S. Van Vleck
Keyword(s):  
Stochastic Games ◽  
Stationary Strategies ◽  
Optimal Stationary Strategies

On zero-sum two-person undiscounted semi-Markov games with a multichain structure

2017 ◽  
Vol 49 (3) ◽  
pp. 826-849 ◽  
Author(s):  
Prasenjit Mondal
Keyword(s):  
Stochastic Game ◽  
Optimal Strategies ◽  
Markov Games ◽  
Stationary Strategies ◽  
Current State ◽  
Strategy Evaluation ◽  
Optimality Equations ◽  
Optimal Stationary Strategies ◽  
Zero Sum ◽  
Absorbing States

Abstract Zero-sum two-person finite undiscounted (limiting ratio average) semi-Markov games (SMGs) are considered with a general multichain structure. We derive the strategy evaluation equations for stationary strategies of the players. A relation between the payoff in the multichain SMG and that in the associated stochastic game (SG) obtained by a data-transformation is established. We prove that the multichain optimality equations (OEs) for an SMG have a solution if and only if the associated SG has optimal stationary strategies. Though the solution of the OEs may not be optimal for an SMG, we establish the significance of studying the OEs for a multichain SMG. We provide a nice example of SMGs in which one player has no optimal strategy in the stationary class but has an optimal semistationary strategy (that depends only on the initial and current state of the game). For an SMG with absorbing states, we prove that solutions in the game where all players are restricted to semistationary strategies are solutions for the unrestricted game. Finally, we prove the existence of stationary optimal strategies for unichain SMGs and conclude that the unichain condition is equivalent to require that the game satisfies some recurrence/ergodicity/weakly communicating conditions.

scholarly journals Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximation

2021 ◽  
Author(s):  
Yue Guan ◽  
Qifan Zhang ◽  
Panagiotis Tsiotras
Keyword(s):  
Nash Equilibrium ◽  
Nash Equilibria ◽  
Stochastic Games ◽  
Computational Cost ◽  
Q Learning ◽  
Scheduling Scheme ◽  
Speed Up ◽  
Zero Sum ◽  
Q Function ◽  
Previous Training

We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the entropy regularization, the algorithm converges to a Nash equilibrium. We also demonstrate the proposed algorithm's ability to transfer previous training experiences, enabling the agents to adapt quickly to new environments. We provide a dynamic hyper-parameter scheduling scheme to further expedite convergence. Empirical results applied to a number of stochastic games verify that the proposed algorithm converges to the Nash equilibrium, while exhibiting a major speed-up over existing algorithms.

scholarly journals Existence of $p$-equilibrium and optimal stationary strategies in stochastic games

1976 ◽  
Vol 60 (1) ◽  
pp. 245-245
Author(s):  
C. J. Himmelberg ◽  
T. Parthasarathy ◽  
T. E. S. Raghavan ◽  
F. S. Van Vleck
Keyword(s):  
Stochastic Games ◽  
Stationary Strategies ◽  
Optimal Stationary Strategies
Sign in / Sign up

Export Citation Format

Share Document