Alternation Measures for the Evaluation of Selfish Agents’ Turn-Taking

Multi-Agent Reinforcement Learning reductionist simulations can provide a spectrum of opportunities towards the modeling and understanding of complex social phenomena such as common-pool appropriation. In this paper, a multiplayer variant of Battle-of-the-Exes is suggested as appropriate for experimentation regarding fair and efficient coordination and turn-taking among selfish agents. Going beyond literature’s fairness and efficiency, a novel measure is proposed for turn-taking coordination evaluation, robust to the number of agents and episodes of a system. Six variants of this measure are defined, entitled Alternation Measures or ALT. ALT measures were found sufficient to capture the desired properties (alternation, fair and efficient distribution) in comparison to state-of-the-art measures, thus they were benchmarked and tested through a series of experiments with Reinforcement Learning agents, aspiring to contribute novel tools for a deeper understanding of emergent social outcomes.

Collective Strategies With a Master-Slave Mechanism Dominate in Spatial-Iterated Prisoner's Dilemma

How cooperation emerges and persists in a population of selfish agents is a fundamental question in evolutionary game theory. The research shows that collective strategies with master-slave mechanism (CSMSM) defeat tit-for-tat and other well-known strategies in spatial iterated prisoner's dilemma. A CSMSM identifies kin members by means of a handshaking mechanism. If the opponent is identified as non-kin, a CSMSM will always defect. Once two CSMSMs meet, they play master and slave roles. A mater defects and a slave cooperates in order to maximize the master's payoff. CSMSM outperforms non-collective strategies in spatial IPD even if there is only a small cluster of CSMSMs in the population. The existence and performance of CSMSM in spatial iterated prisoner's dilemma suggests that cooperation first appears and persists in a group of collective agents.

A Distributed Algorithm for Large-Scale Linearly Coupled Resource Allocation Problems with Selfish Agents

A decentralized randomized coordinate descent method is proposed to solve a large-scale linearly constrained, separable resource optimization problem with selfish agent. This method has a cheap computational cost and can guarantee an improvement of selected objective function without jeopardizing the others in each iteration. The convergence rate is obtained using an alternative gap benchmark of objective value. Numerical simulations suggest that the algorithm will converge to a random point on the Pareto front.

Strategyproof Mechanisms for Additively Separable and Fractional Hedonic Games

Additively separable hedonic games and fractional hedonic games have received considerable attention in the literature. They are coalition formation games among selfish agents based on their mutual preferences. Most of the work in the literature characterizes the existence and structure of stable outcomes (i.e., partitions into coalitions) assuming that preferences are given. However, there is little discussion of this assumption. In fact, agents receive different utilities if they belong to different coalitions, and thus it is natural for them to declare their preferences strategically in order to maximize their benefit. In this paper we consider strategyproof mechanisms for additively separable hedonic games and fractional hedonic games, that is, partitioning methods without payments such that utility maximizing agents have no incentive to lie about their true preferences. We focus on social welfare maximization and provide several lower and upper bounds on the performance achievable by strategyproof mechanisms for general and specific additive functions. In most of the cases we provide tight or asymptotically tight results. All our mechanisms are simple and can be run in polynomial time. Moreover, all the lower bounds are unconditional, that is, they do not rely on any computational complexity assumptions.

A dynamic over games drives selfish agents to win–win outcomes

Understanding human institutions, animal cultures and other social systems requires flexible formalisms that describe how their members change them from within. We introduce a framework for modelling how agents change the games they participate in. We contrast this between-game ‘institutional evolution’ with the more familiar within-game ‘behavioural evolution’. We model institutional change by following small numbers of persistent agents as they select and play a changing series of games. Starting from an initial game, a group of agents trace trajectories through game space by navigating to increasingly preferable games until they converge on ‘attractor’ games. Agents use their ‘institutional preferences' for game features (such as stability, fairness and efficiency) to choose between neighbouring games. We use this framework to pose a pressing question: what kinds of games does institutional evolution select for; what is in the attractors? After computing institutional change trajectories over the two-player space, we find that attractors have disproportionately fair outcomes, even though the agents who produce them are strictly self-interested and indifferent to fairness. This seems to occur because game fairness co-occurs with the self-serving features these agents do actually prefer. We thus present institutional evolution as a mechanism for encouraging the spontaneous emergence of cooperation among small groups of inherently selfish agents, without space, reputation, repetition, or other more familiar mechanisms. Game space trajectories provide a flexible, testable formalism for modelling the interdependencies of behavioural and institutional evolutionary processes, as well as a mechanism for the evolution of cooperation.

Inducing selfish agents towards social efficient solutions

Many multi-agent reinforcement learning (MARL) scenarios lead towards Nash equilibria, which is known to not always be socially efficient. In this study we aim to align the social optimization objective of the system with the individual objectives of the agents by adopting a central controller which can interact with the agents. In details, our approach establishes a communication channel between reinforcement learning agents, and a controller implemented with metaheuristics. The interaction benefit the convergence of both algorithms. Further, we evaluate our method in repeated games with high price of anarchy and show that our approach is able to overcome much of the issues caused by the non-cooperative behaviour of the agents and the non-stationary effects they cause.

Selfish algorithm and emergence of collective intelligence

We propose a model for demonstrating spontaneous emergence of collective intelligent behaviour (i.e. adaptation and resilience of a social system) from selfish individual agents. Agents’ behaviour is modelled using our proposed selfish algorithm ($SA$) with three learning mechanisms: reinforced learning ($SAL$), trust ($SAT$) and connection ($SAC$). Each of these mechanisms provides a distinctly different way an agent can increase the individual benefit accrued through playing the prisoner’s dilemma game ($PDG$) with other agents. $SAL$ generates adaptive reciprocity between the agents with a level of mutual cooperation that depends on the temptation of the individuals to cheat. Adding $SAT$ or $SAC$ to $SAL$ improves the adaptive reciprocity between selfish agents, raising the level of mutual cooperation. Importantly, the mechanisms in the $SA$ are self-tuned by the internal dynamics that depend only on the change in the agent’s own payoffs. This is in contrast to any pre-established reciprocity mechanism (e.g. predefined connections among agents) or awareness of the behaviour or payoffs of other agents. Also, we study adaptation and resilience of the social systems utilizing $SA$ by turning some of the agents to zealots to show that agents reconstruct the reciprocity structure in such a way to eliminate the zealots from getting advantage of a cooperative environment. The implications and applications of the $SA$ are discussed.

HOW SHOULD WE RECONCILE SELF-REGARDING AND PRO-SOCIAL MOTIVATIONS? A RENAISSANCE OF “DAS ADAM SMITH PROBLEM”

Abstract“Das Adam Smith Problem” is the name given by eighteenth-century German scholars to the question of how to reconcile the role of self-interest in the Wealth of Nations with Smith’s advocacy of sympathy in Theory of Moral Sentiments. As the discipline of economics developed, it focused on the interaction of selfish agents, pursuing their private interests. However, behavioral economists have rediscovered the existence and importance of multiple motivations, and a new Das Adam Smith Problem has arisen, of how to accommodate self-regarding and pro-social motivations in a single system. This question is particularly important because of evidence of motivation crowding, where paying people can backfire, with payments achieving the opposite effects of those intended. Psychologists have proposed a mechanism for the crowding out of “intrinsic motivations” for doing a task, when payment is used to incentivize effort. However, they argue that pro-social motivations are different from these intrinsic motivations, implying that crowding out of pro-social motivations requires a different mechanism. In this essay I present an answer to the new Das Adam Smith problem, proposing a mechanism that can underpin the crowding out of both pro-social and intrinsic motivations, whereby motivations are prompted by frames and motivation crowding is underpinned by the crowding out of frames. I explore some of the implications of this mechanism for research and policy.