Games Embedded in Life

Many games focus on a part of the life of an organism. The payoff structure of the game then represents how the game affects fitness proxies such as mean lifetime reproductive success, which are concerned with the whole of the life of the organism. However, the traditional approach of specifying payoffs in advance of the analysis of the game can lead to inconsistencies because the rest of the life of an individual is not fixed but depends on what happens in the game. This chapter concerns this issue, identifying situations in which a more holistic approach is needed. A series of models illustrates links between the current situation and a lifetime perspective. When each of two parents must decide whether to care for their common young or desert, the payoff for desertion depends on the solution of the game and cannot be specified in advance. A game in which two males contest for a female illustrates the approach that must be taken if this game can be repeated at a later time. A game in which individuals must possess territories in order to breed is developed that highlights various interdependencies and, by incorporating learning, advances the understanding of owner–intruder interactions. The interdependencies in state-dependent dynamic games are also illustrated with a model in which individuals must trade off the risks of starvation and predation in a situation in which the choice of the best foraging habitat depends on the number of other animals that use that habitat.

Co-evolution of Traits

Often traits interact, so that considering the evolution of each in isolation gives too limited an account. As is demonstrated in this chapter, it is then crucial to allow for the co-evolution of traits when analysing evolutionary stability. In particular this is important when there there is disruptive selection. Criteria for stability are presented and are applied to a variety of systems. It is shown that role asymmetries can lead to different predictions compared with the analogous situation with the same payoffs but without such asymmetries. Disruptive selection can lead to the evolution of anisogamy. When parental ability and parental effort co-evolve, disruptive selection can lead to one sex evolving to be better at care and doing most of the care. Furthermore, disruptive selection can lead to multiple ESSs, as when prosocial behaviour and the propensity to disperse from the natal site co-evolve. As is shown, disruptive forces can also act when there is learning, leading to specialization. This chapter sets the scene for the chapter that follows, where the level of cooperation shown by individuals co-evolves with choosiness and social sensitivity.

Standard Examples

Standard examples in biological game theory are introduced. The degree of cooperation at evolutionary stability is analysed in models that deal with situations such as the Prisoner’s Dilemma, the Tragedy of the Commons and the conflict of interest between parents over care of their common young. Several models of aggressive interactions are treated in this book. In this chapter the Hawk–Dove game, which is the simplest of these models, is analysed. Further models in the chapter deal with the situation in which individuals vary in their fighting ability and the situation in which information about the opponent is available before an individual decides whether to be aggressive. The problem of the allocation of resources to sons versus daughters has played a central role in biological game theory. This chapter introduces the basic theory, as well as a model in which the environmental temperature affects the development of the sexes differentially, so that at evolutionary stability the sex of offspring is determined by this temperature. Coordination games, alternative mating tactics, dispersal to avoid kin competition, and the idea that signals can evolve from cues are also introduced.

Variation, Consistency, and Reputation

There is typically considerable between-individual variation in trait values in natural populations. Game theory has often ignored this, treating individuals as the same. However, the existence and amount of variation is central to many predictions in biological game theory, as this chapter illustrates. Variation is central to signalling systems and stabilizes these systems as well as extensive-form games. Variation leads to individuals taking a chance that a partner is better than average; for example, promoting cooperation in a finitely repeated Prisoner’s Dilemma game. When there is both variation and within-individual consistency, so that past behaviour is predictive of current behaviour, reputation is important. As is demonstrated, once population members respond to reputation, this then selects for all to modify their behaviour so as to change their reputation and so change how others interact with them in the future, with consequences for the level of cooperation in the population. Furthermore, as a game of trust shows, the extent to which reputation matters can depend on whether individuals are prepared to pay the cost of being socially sensitive, which depends on the amount of variation. Variation selects for individuals to be choosy about their partner, and choosiness can lead to assortative pairing in a population, again promoting cooperation. The importance of choosiness in a market situation is demonstrated by a model in which partners have to decide how much to commit to one another, with factors that enhance choosiness leading to higher levels of commitment.

Stability Concepts: Beyond Nash Equilibria

The concept of an Evolutionarily Stable Strategy (ESS), which is a stronger stability condition than that of a Nash equilibrium, is introduced. A simple evolutionary dynamic, adaptive dynamics, is also introduced. This leads to the concept of convergence stability under adaptive dynamics. It is shown that these two stability criteria are independent for general games: a strategy can be an ESS but not be reachable under adaptive dynamics and a strategy may be an attractor under adaptive dynamics but a fitness minimum and so not an ESS. The latter situation leads to the possibility of evolutionary branching, a phenomenon in which the population splits into a mixture of two or more distinct morphs. Replicator dynamics provide another evolutionary dynamic, although it is argued that it is of limited relevance to biology. In some games, individuals interact with relatives. The effects of kin assortment, and the direct fitness and gene-centred approaches to games between kin are described and illustrated.

Learning in Large Worlds

The chapter introduces reinforcement learning in game-theory models. A distinction is made between small-worlds models with Bayesian updating and large-worlds models that implement specific behavioural mechanisms. The actor–critic learning approach is introduced and illustrated with simple examples of learning in a coordination game and in the Hawk–Dove game. Simple versions of a game of investments with joint benefits and a social dominance game are presented, and these games are further developed in Chapter 8. The idea that parameters of the learning process, such as learning rates, can evolve is put forward. For the game examples it is shown that with slow learning over many rounds the outcome can approximate an ESS of a one-shot game, but for higher rates of learning and fewer rounds this need not be the case. The chapter ends with an overview of learning approaches in game theory, including the originally proposed relative-payoff-sum learning rule for games in biology.

Setting the Scene

The chapter starts with an introduction to game theory in biology, describing its overall aims. The basic concept of frequency dependence is then presented, together with a number of illustrative biological examples. Next, the modelling approach is outlined, emphasizing that the theory aims to predict phenomena by seeking stable evolutionary endpoints. The scope and challenges of the field are described in the setting of the history of ideas that have been important for the theory, summarizing past successes as well as long-standing questions that are likely to require further development of the theory. The chapter ends with an overview of the main issues dealt with in the book, including the challenges that are taken up. These include taking into account the co-evolution of traits, exploring the consequences of variation, and the modelling social interactions as games over time. In particular for the latter, models that include behavioural mechanisms are likely to be essential for the success of game theory in biology.

Structured Populations and Games over Generations

The actions and state of an individual in one generation can affect the state of offspring in the next generation, and hence the ability of these offspring to leave offspring themselves. This chapter deals with games in this multigenerational setting. Projection matrices are used to keep track of the state and number of descendants in successive years and generations. Invasion fitness is then defined in terms of the leading eigenvalue of the projection matrix. Simple examples illustrate these concepts and show how to apply them. Reproductive value is a function that measures how the ability to leave descendants in future generations depends on the current state. The Nash equilibrium condition is reformulated in terms of reproductive value maximization. This new criterion is used to justify the fitness proxy used in the analysis of sex allocation earlier in the book. The analysis is extended to the case where offspring may inherit aspects of their mother’s quality, with a focus on the question of whether high-quality mothers should produce sons or daughters. As a second application of reproductive value maximization, the co-evolution of female preference for a particular male trait and the trait itself is analysed, with the evolution of tail length in the widowbird as an illustrative application. Mean lifetime reproductive success is used as a fitness proxy in much of the book. Its use is finally justified in this chapter, where the fitness proxy is used to analyse the evolutionarily stable age of first reproduction in a population.

Interaction, Negotiation, and Learning

Many social interactions are extended over time, with sequences of decisions by the participating individuals. An interaction can include negotiation between partners as well as learning about each other’s characteristics and qualities. The classical game-theoretical concepts of normal- and extensive-form games and perfect and Stackelberg equilibria are described. A model of the negotiation by parents over investments into their joint offspring is presented and discussed, emphasizing the difference between the game with and without time structure. A version of this model where individuals do not negotiate their investments but instead learn about their respective capacities to invest is presented. For this model a cognitive bias can evolve, such that individuals behave as if they overestimate their true cost of investing. The evolution of social dominance behaviour and social hierarchies is then studied, using actor–critic learning with observations of relative fighting ability and individual recognition. A strong effect of social group size on social dominance is one of the results from the analysis. Pairwise contests are then modelled, both using actor–critic learning and as a sequential assessment game, which is shown to correspond to a neural random walk. The chapter ends with a broad discussion of the successes and challenges of games with time structure. Reasons to incorporate behavioural mechanisms into game theory models, using a large-worlds perspective, are presented.

Central Concepts

The chapter defines and discusses some of the central concepts in biological game theory. Strategies, which are rules for choosing actions as a function of state, play a pivotal role. It is explained how the theory operates at the level of strategies rather than attempting to follow the details of the underlying genetics that code for them. This is referred to as 'the phenotypic gambit', which is discussed and illustrated. The concept of the invasion fitness of a mutant strategy in a population that adopts another resident strategy is also central. This performance measure is used to give a necessary condition for evolutionary stability, formulated as the Nash equilibrium condition. It is explained how this stability condition can be reformulated in terms of simpler fitness proxies such as the mean lifetime number of offspring or the net rate of energy gain.