On the Existence and Determining Stationary Nash Equilibria for Switching Controller Stochastic Games

In this paper we consider the problem of the existence and determining stationary Nash equilibria for switching controller stochastic games with discounted and average payoffs. The set of states and the set of actions in the considered games are assumed to be finite. For a switching controller stochastic game with discounted payoffs we show that all stationary equilibria can be found by using an auxiliary continuous noncooperative static game in normal form in which the payoffs are quasi-monotonic (quasi-convex and quasi-concave) with respect to the corresponding strategies of the players. Based on this we propose an approach for determining the optimal stationary strategies of the players. In the case of average payoffs for a switching controller stochastic game we also formulate an auxiliary noncooperative static game in normal form with quasi-monotonic payoffs and show that such a game possesses a Nash equilibrium if the corresponding switching controller stochastic game has a stationary Nash equilibrium.

A Characterization of Stationary Nash Equilibria of Single Controller Constrained Stochastic Games

We consider a two player finite state-action general sum single controller constrained stochastic game with both discounted and average cost criteria. We consider the situation where player 1 has subscription-based constraints and player 2, who controls the transition probabilities, has realization-based constraints which can also depend on the strategies of player 1. It is known that a stationary Nash equilibrium for discounted case exists under strong Slater condition, while, for the average case, stationary Nash equilibrium exists if additionally the Markov chain is unichain. For each case we show that the set of stationary Nash equilibria of this game has one to one correspondence with the set of global minimizers of a certain nonconvex mathematical program. If the constraints of player 2 do not depend on the strategies of player 1, then the mathematical program reduces to a quadratic program. The known linear programs for zero sum games of this class can be obtained as a special case of above quadratic programs.

BLACKWELL OPTIMALITY IN STOCHASTIC GAMES

Blackwell optimality in a finite state-action discounted Markov decision process (MDP) gives an optimal strategy which is optimal for every discount factor close enough to one. In this article we explore this property, which we call as Blackwell–Nash equilibrium, in two player finite state-action discounted stochastic games. A strategy pair is said to be a Blackwell–Nash equilibrium if it is a Nash equilibrium for every discount factor close enough to one. A stationary Blackwell–Nash equilibrium in a stochastic game may not always exist as can be seen from "Big Match" example where a stationary Nash equilibrium does not exist in undiscounted case. For a Single Controller Additive Reward (SC-AR) stochastic game, we show that there exists a stationary deterministic Blackwell–Nash equilibrium which is also a Nash equilibrium for undiscounted case. For general stochastic games, we give some conditions which together are sufficient for any stationary Nash equilibrium of a discounted stochastic game to be a Blackwell–Nash equilibrium and it is also a Nash equilibrium of an undiscounted stochastic game. We illustrate our results on general stochastic games through a variant of the pollution tax model.