scholarly journals An Exact Double-Oracle Algorithm for Zero-Sum Extensive-Form Games with Imperfect Information

2014 ◽  
Vol 51 ◽  
pp. 829-866 ◽  
Author(s):  
B. Bosansky ◽  
C. Kiekintveld ◽  
V. Lisy ◽  
M. Pechoucek
Keyword(s):  
Nash Equilibrium ◽  
Imperfect Information ◽  
Search Algorithm ◽  
Main Idea ◽  
Substantial Improvement ◽  
Extensive Form ◽  
Extensive Form Games ◽  
Solution Algorithms ◽  
Restricted Game ◽  
Zero Sum

Developing scalable solution algorithms is one of the central problems in computational game theory. We present an iterative algorithm for computing an exact Nash equilibrium for two-player zero-sum extensive-form games with imperfect information. Our approach combines two key elements: (1) the compact sequence-form representation of extensive-form games and (2) the algorithmic framework of double-oracle methods. The main idea of our algorithm is to restrict the game by allowing the players to play only selected sequences of available actions. After solving the restricted game, new sequences are added by finding best responses to the current solution using fast algorithms. We experimentally evaluate our algorithm on a set of games inspired by patrolling scenarios, board, and card games. The results show significant runtime improvements in games admitting an equilibrium with small support, and substantial improvement in memory use even on games with large support. The improvement in memory use is particularly important because it allows our algorithm to solve much larger game instances than existing linear programming methods. Our main contributions include (1) a generic sequence-form double-oracle algorithm for solving zero-sum extensive-form games; (2) fast methods for maintaining a valid restricted game model when adding new sequences; (3) a search algorithm and pruning methods for computing best-response sequences; (4) theoretical guarantees about the convergence of the algorithm to a Nash equilibrium; (5) experimental analysis of our algorithm on several games, including an approximate version of the algorithm.

scholarly journals Solving Large Extensive-Form Games with Strategy Constraints

2019 ◽  
Vol 33 ◽  
pp. 1861-1868
Author(s):  
Trevor Davis ◽  
Kevin Waugh ◽  
Michael Bowling
Keyword(s):  
Private Information ◽  
Imperfect Information ◽  
Risk Mitigation ◽  
Solution Concept ◽  
Optimal Strategies ◽  
Linear Constraints ◽  
Convex Constraints ◽  
Extensive Form ◽  
Extensive Form Games ◽  
Large Extensive Form Games

Extensive-form games are a common model for multiagent interactions with imperfect information. In two-player zerosum games, the typical solution concept is a Nash equilibrium over the unconstrained strategy set for each player. In many situations, however, we would like to constrain the set of possible strategies. For example, constraints are a natural way to model limited resources, risk mitigation, safety, consistency with past observations of behavior, or other secondary objectives for an agent. In small games, optimal strategies under linear constraints can be found by solving a linear program; however, state-of-the-art algorithms for solving large games cannot handle general constraints. In this work we introduce a generalized form of Counterfactual Regret Minimization that provably finds optimal strategies under any feasible set of convex constraints. We demonstrate the effectiveness of our algorithm for finding strategies that mitigate risk in security games, and for opponent modeling in poker games when given only partial observations of private information.

scholarly journals Smoothing Method for Approximate Extensive-Form Perfect Equilibrium

2017 ◽  
Author(s):  
Christian Kroer ◽  
Gabriele Farina ◽  
Tuomas Sandholm
Keyword(s):  
Nash Equilibrium ◽  
Imperfect Information ◽  
Large Scale ◽  
Nash Equilibria ◽  
Convex Polytope ◽  
Solution Concept ◽  
Extensive Form ◽  
Game Tree ◽  
Equilibrium Refinements ◽  
Perfect Equilibria

Nash equilibrium is a popular solution concept for solving imperfect-information games in practice. However, it has a major drawback: it does not preclude suboptimal play in branches of the game tree that are not reached in equilibrium. Equilibrium refinements can mend this issue, but have experienced little practical adoption. This is largely due to a lack of scalable algorithms.Sparse iterative methods, in particular first-order methods, are known to be among the most effective algorithms for computing Nash equilibria in large-scale two-player zero-sum extensive-form games. In this paper, we provide, to our knowledge, the first extension of these methods to equilibrium refinements. We develop a smoothing approach for behavioral perturbations of the convex polytope that encompasses the strategy spaces of players in an extensive-form game. This enables one to compute an approximate variant of extensive-form perfect equilibria. Experiments show that our smoothing approach leads to solutions with dramatically stronger strategies at information sets that are reached with low probability in approximate Nash equilibria, while retaining the overall convergence rate associated with fast algorithms for Nash equilibrium. This has benefits both in approximate equilibrium finding (such approximation is necessary in practice in large games) where some probabilities are low while possibly heading toward zero in the limit, and exact equilibrium computation where the low probabilities are actually zero.

scholarly journals An Algorithm for Constructing and Solving Imperfect Recall Abstractions of Large Extensive-Form Games

2017 ◽  
Author(s):  
Jiri Cermak ◽  
Branislav Bošanský ◽  
Viliam Lisý
Keyword(s):  
Relative Size ◽  
Fictitious Play ◽  
Perfect Recall ◽  
Extensive Form ◽  
Extensive Form Games ◽  
Imperfect Recall ◽  
Information Sets ◽  
Information Set ◽  
Zero Sum ◽  
Large Extensive Form Games

We solve large two-player zero-sum extensive-form games with perfect recall. We propose a new algorithm based on fictitious play that significantly reduces memory requirements for storing average strategies. The key feature is exploiting imperfect recall abstractions while preserving the convergence rate and guarantees of fictitious play applied directly to the perfect recall game. The algorithm creates a coarse imperfect recall abstraction of the perfect recall game and automatically refines its information set structure only where the imperfect recall might cause problems. Experimental evaluation shows that our novel algorithm is able to solve a simplified poker game with 7.10^5 information sets using an abstracted game with only 1.8% of information sets of the original game. Additional experiments on poker and randomly generated games suggest that the relative size of the abstraction decreases as the size of the solved games increases.

scholarly journals Large Scale Learning of Agent Rationality in Two-Player Zero-Sum Games

2019 ◽  
Vol 33 ◽  
pp. 6104-6111 ◽  
Author(s):  
Chun Kai Ling ◽  
Fei Fang ◽  
J. Zico Kolter
Keyword(s):  
Real World ◽  
Large Scale ◽  
Scale Up ◽  
Synthetic Data ◽  
Extensive Form ◽  
Extensive Form Games ◽  
Zero Sum Games ◽  
Primal Dual ◽  
End To End ◽  
Zero Sum

With the recent advances in solving large, zero-sum extensive form games, there is a growing interest in the inverse problem of inferring underlying game parameters given only access to agent actions. Although a recent work provides a powerful differentiable end-to-end learning frameworks which embed a game solver within a deep-learning framework, allowing unknown game parameters to be learned via backpropagation, this framework faces significant limitations when applied to boundedly rational human agents and large scale problems, leading to poor practicality. In this paper, we address these limitations and propose a framework that is applicable for more practical settings. First, seeking to learn the rationality of human agents in complex two-player zero-sum games, we draw upon well-known ideas in decision theory to obtain a concise and interpretable agent behavior model, and derive solvers and gradients for end-to-end learning. Second, to scale up to large, real-world scenarios, we propose an efficient first-order primal-dual method which exploits the structure of extensive-form games, yielding significantly faster computation for both game solving and gradient computation. When tested on randomly generated games, we report speedups of orders of magnitude over previous approaches. We also demonstrate the effectiveness of our model on both real-world one-player settings and synthetic data.

scholarly journals Computing Team-Maxmin Equilibria in Zero-Sum Multiplayer Extensive-Form Games

2020 ◽  
Vol 34 (02) ◽  
pp. 2318-2325
Author(s):  
Youzhi Zhang ◽  
Bo An
Keyword(s):  
Solution Space ◽  
Optimal Strategies ◽  
Mixed Integer ◽  
Multiplayer Games ◽  
Extensive Form ◽  
Mixed Integer Linear Program ◽  
Extensive Form Games ◽  
Security Games ◽  
Zero Sum ◽  
Constraint Method

The study of finding the equilibrium for multiplayer games is challenging. This paper focuses on computing Team-Maxmin Equilibria (TMEs) in zero-sum multiplayer Extensive-Form Games (EFGs), which describes the optimal strategies for a team of players who share the same goal but they take actions independently against an adversary. TMEs can capture many realistic scenarios, including: 1) a team of players play against a target player in poker games; and 2) defense resources schedule and patrol independently in security games. However, the study of efficiently finding TMEs within any given accuracy in EFGs is almost completely unexplored. To fill this gap, we first study the inefficiency caused by computing the equilibrium where team players correlate their strategies and then transforming it into the mixed strategy profile of the team and show that this inefficiency can be arbitrarily large. Second, to efficiently solve the non-convex program for finding TMEs directly, we develop the Associated Recursive Asynchronous Multiparametric Disaggregation Technique (ARAMDT) to approximate multilinear terms in the program with two novel techniques: 1) an asynchronous precision method to reduce the number of constraints and variables for approximation by using different precision levels to approximate these terms; and 2) an associated constraint method to reduce the feasible solution space of the mixed-integer linear program resulting from ARAMDT by exploiting the relation between these terms. Third, we develop a novel iterative algorithm to efficiently compute TMEs within any given accuracy based on ARAMDT. Our algorithm is orders of magnitude faster than baselines in the experimental evaluation.

scholarly journals Algorithm for Computing Approximate Nash Equilibrium in Continuous Games with Application to Continuous Blotto

Games ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 47
Author(s):  
Sam Ganzfried
Keyword(s):  
Nash Equilibrium ◽  
Imperfect Information ◽  
Pure Strategy ◽  
Strategy Space ◽  
Equilibrium Strategies ◽  
Approximate Nash Equilibrium ◽  
Blotto Game ◽  
Nash Equilibrium Strategies ◽  
Zero Sum ◽  
Action Spaces

Successful algorithms have been developed for computing Nash equilibrium in a variety of finite game classes. However, solving continuous games—in which the pure strategy space is (potentially uncountably) infinite—is far more challenging. Nonetheless, many real-world domains have continuous action spaces, e.g., where actions refer to an amount of time, money, or other resource that is naturally modeled as being real-valued as opposed to integral. We present a new algorithm for approximating Nash equilibrium strategies in continuous games. In addition to two-player zero-sum games, our algorithm also applies to multiplayer games and games with imperfect information. We experiment with our algorithm on a continuous imperfect-information Blotto game, in which two players distribute resources over multiple battlefields. Blotto games have frequently been used to model national security scenarios and have also been applied to electoral competition and auction theory. Experiments show that our algorithm is able to quickly compute close approximations of Nash equilibrium strategies for this game.

scholarly journals ALTERNATIVE SELECTION FUNCTIONS FOR INFORMATION SET MONTE CARLO TREE SEARCH

2014 ◽  
Vol 54 (5) ◽  
pp. 333-340
Author(s):  
Viliam Lisy
Keyword(s):  
Monte Carlo ◽  
Imperfect Information ◽  
Search Algorithm ◽  
Superior Performance ◽  
Tree Search ◽  
Monte Carlo Tree Search ◽  
Information Set ◽  
Imperfect Information Games ◽  
Zero Sum ◽  
Tree Search Algorithm

We evaluate the performance of various selection methods for the Monte Carlo Tree Search algorithm in two-player zero-sum extensive-form games with imperfect information. We compare the standard Upper Confident Bounds applied to Trees (UCT) along with the less common Exponential Weights for Exploration and Exploitation (Exp3) and novel Regret matching (RM) selection in two distinct imperfect information games: Imperfect Information Goofspiel and Phantom Tic-Tac-Toe. We show that UCT after initial fast convergence towards a Nash equilibrium computes increasingly worse strategies after some point in time. This is not the case with Exp3 and RM, which also show superior performance in head-to-head matches.

Sign in / Sign up

Export Citation Format

Share Document