Trans-National Equitable Strategies of Vaccine Distribution during the COVID-19 Pandemic

In this report we propose a modelling framework to analyse and optimise worldwide vaccine allocation strategies, with reference to the current COVID-19 pandemic. We model epidemiological transmission and vaccination in a system of M interacting countries, quantifying the social and economic costs incurred by each country due to the pandemic and the country's vaccination rate. Under constraints over global vaccine availability, we analyse best strategies of vaccine allocation: either with the aim of minimising global costs, or by taking the perspective of a competitive game where each country aims to minimise its own cost. We distinguish between the financial capabilities of different countries, and extend our framework to allow for vaccine donation from wealthier to poorer countries. Numerical simulations allow us to compare the best strategies of the above two approaches, and to analyse circumstances under which vaccine donation simultaneously benefits both donating and receiving countries.

Trans-National Equitable Strategies of Vaccine Distribution during the COVID-19 Pandemic

In this report we propose a modelling framework to analyse and optimise worldwide vaccine allocation strategies, with reference to the current COVID-19 pandemic. We model epidemiological transmission and vaccination in a system of M interacting countries, quantifying the social and economic costs incurred by each country due to the pandemic and the country's vaccination rate. Under constraints over global vaccine availability, we analyse best strategies of vaccine allocation: either with the aim of minimising global costs, or by taking the perspective of a competitive game where each country aims to minimise its own cost. We distinguish between the financial capabilities of different countries, and extend our framework to allow for vaccine donation from wealthier to poorer countries. Numerical simulations allow us to compare the best strategies of the above two approaches, and to analyse circumstances under which vaccine donation simultaneously benefits both donating and receiving countries.

Characterizing human random-sequence generation in competitive and non-competitive environments using Lempel–Ziv complexity

The human ability for random-sequence generation (RSG) is limited but improves in a competitive game environment with feedback. However, it remains unclear how random people can be during games and whether RSG during games can improve when explicitly informing people that they must be as random as possible to win the game. Nor is it known whether any such improvement in RSG transfers outside the game environment. To investigate this, we designed a pre/post intervention paradigm around a Rock-Paper-Scissors game followed by a questionnaire. During the game, we manipulated participants’ level of awareness of the computer’s strategy; they were either (a) not informed of the computer’s algorithm or (b) explicitly informed that the computer used patterns in their choice history against them, so they must be maximally random to win. Using a compressibility metric of randomness, our results demonstrate that human RSG can reach levels statistically indistinguishable from computer pseudo-random generators in a competitive-game setting. However, our results also suggest that human RSG cannot be further improved by explicitly informing participants that they need to be random to win. In addition, the higher RSG in the game setting does not transfer outside the game environment. Furthermore, we found that the underrepresentation of long repetitions of the same entry in the series explains up to 29% of the variability in human RSG, and we discuss what might make up the variance left unexplained.

Characterizing human random-sequence generation in competitive and non-competitive environments using Lempel-Ziv complexity

The human ability for random-sequence generation (RSG) is limited but improves in a competitive game environment with feedback. However, it remains unclear how random people can be during games and whether RSG during games can improve when explicitly informing people that they must be as random as possible to win the game. Nor is it known whether any such improvement in RSG transfers outside the game environment. To investigate this, we designed a pre/post intervention paradigm around a Rock-Paper-Scissors game followed by a questionnaire. During the game, we manipulated participants’ level of awareness of the computer’s strategy; they were either (a) not informed of the computer’s algorithm or (b) explicitly informed that the computer used patterns in their choice history against them, so they must be maximally random to win. Using a compressibility metric of randomness, our results demonstrate that human RSG can reach levels statistically indistinguishable from computer pseudo- random generators in a competitive-game setting. However, our results also suggest that human RSG cannot be further improved by explicitly informing participants that they need to be random to win. In addition, the higher RSG in the game setting does not transfer outside the game environment. Furthermore, we found that the underrepresentation of long repetitions of the same entry in the series explains up to 29% of the variability in human RSG, and we discuss what might make up the variance left unexplained.

Pupil correlates of decision variables in mice playing a competitive mixed-strategy game

In a competitive game involving an animal and an opponent, the outcome is contingent on the choices of both players. To succeed, the animal must continually adapt to competitive pressure, or else risk being exploited and lose out on rewards. In this study, we demonstrate that head-fixed mice can be trained to play the iterative competitive game 'matching pennies' against a virtual computer opponent. We find that the animals' performance is well described by a hybrid computational model that includes Q-learning and choice kernels. Comparing between matching pennies and a non-competitive two-armed bandit task, we show that the tasks encourage animals to operate at different regimes of reinforcement learning. To understand the involvement of neuromodulatory mechanisms, we measure fluctuations in pupil size and use multiple linear regression to relate the trial-by-trial transient pupil responses to decision-related variables. The analysis reveals that pupil responses are modulated by observable variables, including choice and outcome, as well as latent variables for value updating, but not action selection. Collectively, these results establish a paradigm for studying competitive decision-making in head-fixed mice and provide insights into the role of arousal-linked neuromodulation in the decision process.

Goal Recognition for Deceptive Human Agents through Planning and Gaze

Eye gaze has the potential to provide insight into the minds of individuals, and this idea has been used in prior research to improve human goal recognition by combining human's actions and gaze. However, most existing research assumes that people are rational and honest. In adversarial scenarios, people may deliberately alter their actions and gaze, which presents a challenge to goal recognition systems. In this paper, we present new models for goal recognition under deception using a combination of gaze behaviour and observed movements of the agent. These models aim to detect when a person is deceiving by analysing their gaze patterns and use this information to adjust the goal recognition. We evaluated our models in two human-subject studies: (1) using data collected from 30 individuals playing a navigation game inspired by an existing deception study and (2) using data collected from 40 individuals playing a competitive game (Ticket To Ride). We found that one of our models (Modulated Deception Gaze+Ontic) offers promising results compared to the previous state-of-the-art model in both studies. Our work complements existing adversarial goal recognition systems by equipping these systems with the ability to tackle ambiguous gaze behaviours.

You Were Always on My Mind: Introducing Chef’s Hat and COPPER for Personalized Reinforcement Learning

Reinforcement learning simulation environments pose an important experimental test bed and facilitate data collection for developing AI-based robot applications. Most of them, however, focus on single-agent tasks, which limits their application to the development of social agents. This study proposes the Chef’s Hat simulation environment, which implements a multi-agent competitive card game that is a complete reproduction of the homonymous board game, designed to provoke competitive strategies in humans and emotional responses. The game was shown to be ideal for developing personalized reinforcement learning, in an online learning closed-loop scenario, as its state representation is extremely dynamic and directly related to each of the opponent’s actions. To adapt current reinforcement learning agents to this scenario, we also developed the COmPetitive Prioritized Experience Replay (COPPER) algorithm. With the help of COPPER and the Chef’s Hat simulation environment, we evaluated the following: (1) 12 experimental learning agents, trained via four different regimens (self-play, play against a naive baseline, PER, or COPPER) with three algorithms based on different state-of-the-art learning paradigms (PPO, DQN, and ACER), and two “dummy” baseline agents that take random actions, (2) the performance difference between COPPER and PER agents trained using the PPO algorithm and playing against different agents (PPO, DQN, and ACER) or all DQN agents, and (3) human performance when playing against two different collections of agents. Our experiments demonstrate that COPPER helps agents learn to adapt to different types of opponents, improving the performance when compared to off-line learning models. An additional contribution of the study is the formalization of the Chef’s Hat competitive game and the implementation of the Chef’s Hat Player Club, a collection of trained and assessed agents as an enabler for embedding human competitive strategies in social continual and competitive reinforcement learning.

Avoiding competition: the effects of rankings in the food waste field

Ranks are often seen as drivers for competition. This chapter critically examines the link between ranks and competition by investigating the actors’ actual desire for the highest positions. Empirically, the author examines the role of the food waste hierarchy in establishing status competition in the food waste field. This discrete ranking creates ‘winners’ at the top (the challengers that prevent food waste by generating demand for it), who respond enthusiastically to the food waste hierarchy to benefit from status gains. In contrast, the ‘losers’ at the bottom (biogas plants) show very little interest in improving their position. They do not see themselves as players in such a competitive game and direct their attention towards other competitions outside the field. The chapter argues that ranks do not necessarily induce competition, since the actors may be involved in multiple competitions and decide whether it is worth pursuing high status within each one.