Temporal Induced Self-Play for Stochastic Bayesian Games

One practical requirement in solving dynamic games is to ensure that the players play well from any decision point onward. To satisfy this requirement, existing efforts focus on equilibrium refinement, but the scalability and applicability of existing techniques are limited. In this paper, we propose Temporal-Induced Self-Play (TISP), a novel reinforcement learning-based framework to find strategies with decent performances from any decision point onward. TISP uses belief-space representation, backward induction, policy learning, and non-parametric approximation. Building upon TISP, we design a policy-gradient-based algorithm TISP-PG. We prove that TISP-based algorithms can find approximate Perfect Bayesian Equilibrium in zero-sum one-sided stochastic Bayesian games with finite horizon. We test TISP-based algorithms in various games, including finitely repeated security games and a grid-world game. The results show that TISP-PG is more scalable than existing mathematical programming-based methods and significantly outperforms other learning-based methods.

Falsification of Hybrid Systems Using Adaptive Probabilistic Search

We present and analyse an algorithm that quickly finds falsifying inputs for hybrid systems. Our method is based on a probabilistically directed tree search, whose distribution adapts to consider an increasingly fine-grained discretization of the input space. In experiments with standard benchmarks, our algorithm shows comparable or better performance to existing techniques, yet it does not build an explicit model of a system. Instead, at each decision point within a single trial, it makes an uninformed probabilistic choice between simple strategies to extend the input signal by means of exploration or exploitation. Key to our approach is the way input signal space is decomposed into levels, such that coarse segments are more probable than fine segments. We perform experiments to demonstrate how and why our approach works, finding that a fully randomized exploration strategy performs as well as our original algorithm that exploits robustness. We propose this strategy as a new baseline for falsification and conclude that more discriminative benchmarks are required.

Britain and the United States Neutralize Spain, 1940–41

This chapter explores Britain and the United States' success in keeping Spain from joining the Axis in 1940 to 1941. Spain nearly entered World War II on the Axis side in late 1940. Yet, at the most crucial decision point — in December of 1940 — Spain's dictator Francisco Franco rebuffed the Axis alliance offers. Franco's decision was due in large part to an Anglo-American effort — initiated and led by Britain — to use inducements to keep Spain sidelined. When Hitler and Franco were converging on an alliance, Spain desperately needed not just military support to fight, but also economic aid — to recover from the civil war, survive severe shortages of food, and secure other basic economic necessities. The concerted Anglo-American policy convinced Franco that Spain's economic needs could best be met through British and U.S. largesse, which could only be obtained if Spain remained nonbelligerent. The British and Americans agreed on the goal (to keep Spain nonbelligerent), on the way to achieve it (inducements), and most of all, about Spain's high strategic weight. This produced a powerful wedge strategy, because the duo was in a good position to influence Spain through coordinated inducements.