BLACKWELL OPTIMALITY IN STOCHASTIC GAMES

2013 ◽  
Vol 15 (04) ◽  
pp. 1340025
Author(s):  
VIKAS VIKRAM SINGH ◽  
N. HEMACHANDRA ◽  
K. S. MALLIKARJUNA RAO
Keyword(s):  
Nash Equilibrium ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Discount Factor ◽  
State Action ◽  
Blackwell Optimality ◽  
Finite State ◽  
Markov Decision ◽  
Stationary Nash Equilibrium ◽  
General Stochastic

Blackwell optimality in a finite state-action discounted Markov decision process (MDP) gives an optimal strategy which is optimal for every discount factor close enough to one. In this article we explore this property, which we call as Blackwell–Nash equilibrium, in two player finite state-action discounted stochastic games. A strategy pair is said to be a Blackwell–Nash equilibrium if it is a Nash equilibrium for every discount factor close enough to one. A stationary Blackwell–Nash equilibrium in a stochastic game may not always exist as can be seen from "Big Match" example where a stationary Nash equilibrium does not exist in undiscounted case. For a Single Controller Additive Reward (SC-AR) stochastic game, we show that there exists a stationary deterministic Blackwell–Nash equilibrium which is also a Nash equilibrium for undiscounted case. For general stochastic games, we give some conditions which together are sufficient for any stationary Nash equilibrium of a discounted stochastic game to be a Blackwell–Nash equilibrium and it is also a Nash equilibrium of an undiscounted stochastic game. We illustrate our results on general stochastic games through a variant of the pollution tax model.

scholarly journals On the Existence and Determining Stationary Nash Equilibria for Switching Controller Stochastic Games

2021 ◽  
Vol 14 ◽  
pp. 290-301
Author(s):  
Dmitrii Lozovanu ◽  
◽  
Stefan Pickl ◽  
Keyword(s):  
Nash Equilibrium ◽  
Normal Form ◽  
Nash Equilibria ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Static Game ◽  
Stationary Strategies ◽  
Switching Controller ◽  
Optimal Stationary Strategies ◽  
Stationary Nash Equilibrium

In this paper we consider the problem of the existence and determining stationary Nash equilibria for switching controller stochastic games with discounted and average payoffs. The set of states and the set of actions in the considered games are assumed to be finite. For a switching controller stochastic game with discounted payoffs we show that all stationary equilibria can be found by using an auxiliary continuous noncooperative static game in normal form in which the payoffs are quasi-monotonic (quasi-convex and quasi-concave) with respect to the corresponding strategies of the players. Based on this we propose an approach for determining the optimal stationary strategies of the players. In the case of average payoffs for a switching controller stochastic game we also formulate an auxiliary noncooperative static game in normal form with quasi-monotonic payoffs and show that such a game possesses a Nash equilibrium if the corresponding switching controller stochastic game has a stationary Nash equilibrium.

A Characterization of Stationary Nash Equilibria of Single Controller Constrained Stochastic Games

2015 ◽  
Vol 17 (02) ◽  
pp. 1540018
Author(s):  
Vikas Vikram Singh ◽  
N. Hemachandra
Keyword(s):  
Nash Equilibrium ◽  
Nash Equilibria ◽  
Mathematical Program ◽  
Transition Probabilities ◽  
Stochastic Game ◽  
Quadratic Program ◽  
Linear Programs ◽  
Average Case ◽  
Single Controller ◽  
Stationary Nash Equilibrium

We consider a two player finite state-action general sum single controller constrained stochastic game with both discounted and average cost criteria. We consider the situation where player 1 has subscription-based constraints and player 2, who controls the transition probabilities, has realization-based constraints which can also depend on the strategies of player 1. It is known that a stationary Nash equilibrium for discounted case exists under strong Slater condition, while, for the average case, stationary Nash equilibrium exists if additionally the Markov chain is unichain. For each case we show that the set of stationary Nash equilibria of this game has one to one correspondence with the set of global minimizers of a certain nonconvex mathematical program. If the constraints of player 2 do not depend on the strategies of player 1, then the mathematical program reduces to a quadratic program. The known linear programs for zero sum games of this class can be obtained as a special case of above quadratic programs.

scholarly journals Policy Invariance under Reward Transformations for General-Sum Stochastic Games

2011 ◽  
Vol 41 ◽  
pp. 397-406 ◽  
Author(s):  
X. Lu ◽  
H. M. Schwartz ◽  
S. N. Givigi
Keyword(s):  
Markov Decision Processes ◽  
Nash Equilibria ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Decision Processes ◽  
Markov Decision

We extend the potential-based shaping method from Markov decision processes to multi-player general-sum stochastic games. We prove that the Nash equilibria in a stochastic game remains unchanged after potential-based shaping is applied to the environment. The property of policy invariance provides a possible way of speeding convergence when learning to play a stochastic game.

MARKOV STRATEGIES ARE BETTER THAN STATIONARY STRATEGIES

1999 ◽  
Vol 01 (01) ◽  
pp. 9-31 ◽  
Author(s):  
J. FLESCH ◽  
F. THUIJSMAN ◽  
O. J. VRIEZE
Keyword(s):  
Stochastic Games ◽  
Stochastic Game ◽  
Initial State ◽  
Stationary Strategies ◽  
Finite State ◽  
Zero Sum ◽  
Action Spaces ◽  
Markov Strategies ◽  
Better Than

We examine the use of stationary and Markov strategies in zero-sum stochastic games with finite state and action spaces. It is natural to evaluate a strategy for the maximising player, player 1, by the highest reward guaranteed to him against any strategy of the opponent. The highest rewards guaranteed by stationary strategies or by Markov strategies are called the stationary utility or the Markov utility, respectively. Since all stationary strategies are Markov strategies, the Markov utility is always larger or equal to the stationary utility. However, in all presently known subclasses of stochastic games, these utilities turn out to be equal. In this paper, we provide a colourful example in which the Markov utility is strictly larger than the stationary utility and we present several conditions under which the utilities are equal. We also show that each stochastic game has at least one initial state for which the two utilities are equal. Several examples clarify these issues.

scholarly journals Cooperative Stochastic Games with Mean-Variance Preferences

Mathematics ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 230
Author(s):  
Elena Parilina ◽  
Stepan Akimochkin
Keyword(s):  
Standard Deviation ◽  
Stochastic Games ◽  
Stochastic Game ◽  
Stochastic Variable ◽  
Expected Payoff ◽  
The Core ◽  
Risk Sensitive ◽  
Payoff Functions ◽  
Cooperative Solutions ◽  
Mean Variance

In stochastic games, the player’s payoff is a stochastic variable. In most papers, expected payoff is considered as a payoff, which means the risk neutrality of the players. However, there may exist risk-sensitive players who would take into account “risk” measuring their stochastic payoffs. In the paper, we propose a model of stochastic games with mean-variance payoff functions, which is the sum of expectation and standard deviation multiplied by a coefficient characterizing a player’s attention to risk. We construct a cooperative version of a stochastic game with mean-variance preferences by defining characteristic function using a maxmin approach. The imputation in a cooperative stochastic game with mean-variance preferences is supposed to be a random vector. We construct the core of a cooperative stochastic game with mean-variance preferences. The paper extends existing models of discrete-time stochastic games and approaches to find cooperative solutions in these games.

Sign in / Sign up

Export Citation Format

Share Document