Nash equilibria in human sensorimotor interactions explained by Q-learning with intrinsic costs

The Nash equilibrium concept has previously been shown to be an important tool to understand human sensorimotor interactions, where different actors vie for minimizing their respective effort while engaging in a multi-agent motor task. However, it is not clear how such equilibria are reached. Here, we compare different reinforcement learning models to human behavior engaged in sensorimotor interactions with haptic feedback based on three classic games, including the prisoner’s dilemma, and the symmetric and asymmetric matching pennies games. We find that a discrete analysis that reduces the continuous sensorimotor interaction to binary choices as in classical matrix games does not allow to distinguish between the different learning algorithms, but that a more detailed continuous analysis with continuous formulations of the learning algorithms and the game-theoretic solutions affords different predictions. In particular, we find that Q-learning with intrinsic costs that disfavor deviations from average behavior explains the observed data best, even though all learning algorithms equally converge to admissible Nash equilibrium solutions. We therefore conclude that it is important to study different learning algorithms for understanding sensorimotor interactions, as such behavior cannot be inferred from a game-theoretic analysis alone, that simply focuses on the Nash equilibrium concept, as different learning algorithms impose preferences on the set of possible equilibrium solutions due to the inherent learning dynamics.

Polycomb condensates can promote epigenetic marks but are not required for sustained chromatin compaction

Organization of the genome into transcriptionally active euchromatin and silenced heterochromatin is essential for eukaryotic cell function. Phase-separation has been implicated in heterochromatin formation, but it is unclear how phase-separated condensates can contribute to stable repression, particularly for heritable epigenetic changes. Polycomb complex PRC1 is key for heterochromatin formation, but the multitude of Polycomb proteins has hindered our understanding of their collective contribution to chromatin repression. Here, we show that PRC1 forms multicomponent condensates through hetero-oligomerization. They preferentially seed at H3K27me3 marks, and subsequently write H2AK119Ub marks. We show that inducing Polycomb phase-separation can cause chromatin compaction, but polycomb condensates are dispensable for maintenance of the compacted state. Our data and simulations are consistent with a model in which the time integral of Polycomb phase-separation is progressively recorded in repressive histone marks, which subsequently drive compaction. These findings link the equilibrium thermodynamics of phase-separation with the fundamentally non-equilibrium concept of epigenetic memory.

Entrepreneurship and Equilibrium

Neoclassic economic theory regards equilibrium, whether general or partial, as a crucially important foundation of the dismal science[1]. In the view of mainstream economists, the general equilibrium framework not only an investigation of the economy in terms of its perfect qualities, but also is suitable as an end or goal of action. The Austrian school, in contrast, sees equilibrium (or the evenly rotating economy - ERE) merely as a tendency for economic activities to move us in that direction, but it is never attained. Praxeological economics has thus concentrated not on equilibrium, but on the process by which the market moves toward it. Since the process of shifting resources to meet these ends cannot be achieved spontaneously, entrepreneurship plays a key role. In Mises and Rothbard’s view, entrepreneurship involves uncertainty bearing which beyond the alertness emphasized by Hayek and Kirzner; on the other hand, unlike Lachmann looks the economy as a kaleidoscope and rejects the ERE, Mises and Rothbard regard the ERE as an indispensable way to understand the economy. [1] As Frank H. Hahn said “Whatever economics is used or thought about, equilibrium is a central organizing concept.” (Hahn 1984: 43). Many economists and philosophers of science consider mathematical neoclassical general equilibrium theory as one of the peak achievements of economics (Rosenberg, 1992). Tieben (2012) stated that policy-makers and theorists of all schools of economics all use some form of equilibrium theory to develop their ideas and support their main theoretical and political claims. Lawson (2005) indicated that the equilibrium concept is a major cause of controversy between different schools of economic thought. A.W. Bob Coats even deems it the case that “economics has been dominated throughout its history by a single paradigm – the theory of economic equilibrium via the market mechanism.” (Coats 1969: 292).

Nash equilibria in human sensorimotor interactions explained by Q-Learning

The Nash equilibrium concept has previously been shown to be an important tool to understand human sensorimotor interactions, where different actors vie for minimizing their respective effort while engaging in a multi-agent motor task. However, it is not clear how such equilibria are reached. Here, we compare different reinforcement learning models based on haptic feedback to human behavior in sensorimotor versions of three classic games, including the Prisoner's Dilemma, and the symmetric and asymmetric matching pennies games. We find that a discrete analysis that reduces the continuous sensorimotor interaction to binary choices as in classical matrix games does not allow to distinguish between the different learning algorithms, but that a more detailed continuous analysis with continuous formulations of the learning algorithms and the game-theoretic solutions affords different predictions. In particular, we find that Q-learning with intrinsic costs that disfavor deviations from average behavior explains the observed data best, even though all learning algorithms equally converge to admissible Nash equilibrium solutions. We therefore conclude that it is important to study different learning algorithms for understanding sensorimotor interactions, as such behavior cannot be inferred from a game-theoretic analysis alone, that simply focuses on the Nash equilibrium concept, as different learning algorithms impose preferences on the set of possible equilibrium solutions due to the inherent learning dynamics.

Self-Organized Criticality in Economic Fluctuations: The Age of Maturity

Self-Organized Criticality (SOC) has been proposed as a paradigm that may rationalize the emergence of macrofinancial fluctuations. The wave of innovative thinking sparked by this proposal continues to produce interesting contributions in many areas of economics, ranging from macroeconomics to finance. In this review, we propose a guided tour to these achievements, highlighting that analysis of SOC equilibria is a promising avenue to establish a nexus between i) a statistical equilibrium characterized by the spontaneous emergence of dynamic critical fluctuations and ii) a strategic equilibrium concept modeling a large number of interacting players. The critical state is the stable outcome arising from a trade-off between cooperation and competition.

Holding a group together: non-game-theory vs. game-theory

Each member of a group chooses a position and has preferences regarding his chosen position. The group’s harmony depends on the profile of chosen positions meeting a specific condition. We analyse a solution concept (Richter and Rubinstein, 2020) based on a permissible set of individual positions, which plays a role analogous to that of prices in competitive equilibrium. Given the permissible set, members choose their most preferred position. The set is tightened if the chosen positions are inharmonious and relaxed if the restrictions are unnecessary. This new equilibrium concept yields more attractive outcomes than does Nash equilibrium in the corresponding game.

Biased preferences equilibrium

We model economic environments in which individual choice sets are fixed and the level of a specific parameter that systematically modifies the preferences of all agents is determined endogenously to achieve equilibrium. The equilibrium concept, Biased Preferences Equilibrium, is reminiscent of competitive equilibrium: agents’ choice sets and their preferences are independent of the behaviour of other agents, the combined choices must satisfy overall feasibility constraints and the endogenous adjustment of the equilibrating preference parameter is analogous to equilibrating price adjustment. The concept is applied in a number of economic examples.

The Equilibrium Concept, or…(Mis)concept in Beaches

Beaches, as deposits of unconsolidated material at the land/water interface, are open systems where input and output items constitute the sediment budget. Beach evolution depends on the difference between the input/output to the system; if positive the beach advances, if negative the beach retreats. Is it possible that this difference is zero and the beach is stable? The various processes responsible for sediment input and output in any beach system are here considered by taking examples from the literature. Results show that this can involve movement of a volume of sediments ranging from few, to over a million cubic meters per year, with figures continuously changing so that the statistical possibility for the budget being equal can be considered zero. This can be attributed to the fact that very few processes are feedback-regulated, which is the only possibility for a natural system to be in equilibrium. Usage of the term “beach equilibrium” must be reconsidered and used with great caution.