Computing Nash Equilibria in Multiplayer DAG-Structured Stochastic Games with Persistent Imperfect Information

Controlled by several agents, multistage processes of resource allocation between production and consumption with random production functions are considered as non-cooperative dynamic stochastic games. For these games, the Nash Equilibria are constructed satisfying the criteria of maximisation of some kind of "public utility". Both finite and infinite horizons of planning are examined.

Download Full-text

Smoothing Method for Approximate Extensive-Form Perfect Equilibrium

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/42 ◽

2017 ◽

Cited By ~ 2

Author(s):

Christian Kroer ◽

Gabriele Farina ◽

Tuomas Sandholm

Keyword(s):

Nash Equilibrium ◽

Imperfect Information ◽

Large Scale ◽

Nash Equilibria ◽

Convex Polytope ◽

Solution Concept ◽

Extensive Form ◽

Game Tree ◽

Equilibrium Refinements ◽

Perfect Equilibria

Nash equilibrium is a popular solution concept for solving imperfect-information games in practice. However, it has a major drawback: it does not preclude suboptimal play in branches of the game tree that are not reached in equilibrium. Equilibrium refinements can mend this issue, but have experienced little practical adoption. This is largely due to a lack of scalable algorithms.Sparse iterative methods, in particular first-order methods, are known to be among the most effective algorithms for computing Nash equilibria in large-scale two-player zero-sum extensive-form games. In this paper, we provide, to our knowledge, the first extension of these methods to equilibrium refinements. We develop a smoothing approach for behavioral perturbations of the convex polytope that encompasses the strategy spaces of players in an extensive-form game. This enables one to compute an approximate variant of extensive-form perfect equilibria. Experiments show that our smoothing approach leads to solutions with dramatically stronger strategies at information sets that are reached with low probability in approximate Nash equilibria, while retaining the overall convergence rate associated with fast algorithms for Nash equilibrium. This has benefits both in approximate equilibrium finding (such approximation is necessary in practice in large games) where some probabilities are low while possibly heading toward zero in the limit, and exact equilibrium computation where the low probabilities are actually zero.

Download Full-text

On the existence of stationary Nash equilibria in average stochastic games with finite state and action spaces

Contributions to Game Theory and Management ◽

10.21638/11701/spbu31.2020.16 ◽

2020 ◽

Vol 13 ◽

pp. 304-323

Author(s):

Dmitrii Lozovanu ◽

◽

Stefan Pickl ◽

Keyword(s):

Nash Equilibria ◽

Stochastic Games ◽

Optimal Strategies ◽

Stationary Strategies ◽

Finite State ◽

Optimal Stationary Strategies ◽

Action Spaces

We consider infinite n-person stochastic games with limiting average payoffs criteria for the players. The main results of the paper are concerned with the existence of stationary Nash equilibria and determining the optimal strategies of the players in the games with finite state and action spaces. We present conditions for the existence of stationary Nash equilibria in the considered games and propose an approach for determining the optimal stationary strategies of the players if such strategies exist.

Download Full-text

Nonzero-Sum Stochastic Games and Mean-Field Games with Impulse Controls

Mathematics of Operations Research ◽

10.1287/moor.2021.1131 ◽

2021 ◽

Author(s):

Matteo Basei ◽

Haoyang Cao ◽

Xin Guo

Keyword(s):

Nash Equilibria ◽

Stochastic Games ◽

Mean Field ◽

Sufficient Conditions ◽

Mean Field Games ◽

Model Parameters ◽

Management Problem ◽

Player Game ◽

The Impact ◽

Impulse Controls

We consider a general class of nonzero-sum N-player stochastic games with impulse controls, where players control the underlying dynamics with discrete interventions. We adopt a verification approach and provide sufficient conditions for the Nash equilibria (NEs) of the game. We then consider the limiting situation when N goes to infinity, that is, a suitable mean-field game (MFG) with impulse controls. We show that under appropriate technical conditions, there exists a unique NE solution to the MFG, which is an ϵ-NE approximation to the N-player game, with [Formula: see text]. As an example, we analyze in detail a class of two-player stochastic games which extends the classical cash management problem to the game setting. In particular, we present numerical analysis for the cases of the single player, the two-player game, and the MFG, showing the impact of competition on the player’s optimal strategy, with sensitivity analysis of the model parameters.

Download Full-text

Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximation

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/339 ◽

2021 ◽

Author(s):

Yue Guan ◽

Qifan Zhang ◽

Panagiotis Tsiotras

Keyword(s):

Nash Equilibrium ◽

Nash Equilibria ◽

Stochastic Games ◽

Computational Cost ◽

Q Learning ◽

Scheduling Scheme ◽

Speed Up ◽

Zero Sum ◽

Q Function ◽

Previous Training

We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the entropy regularization, the algorithm converges to a Nash equilibrium. We also demonstrate the proposed algorithm's ability to transfer previous training experiences, enabling the agents to adapt quickly to new environments. We provide a dynamic hyper-parameter scheduling scheme to further expedite convergence. Empirical results applied to a number of stochastic games verify that the proposed algorithm converges to the Nash equilibrium, while exhibiting a major speed-up over existing algorithms.

Download Full-text