scholarly journals Mean Field Games Flock! The Reinforcement Learning Way

2021 ◽  
Author(s):  
Sarah Perrin ◽  
Mathieu Laurière ◽  
Julien Pérolat ◽  
Matthieu Geist ◽  
Romuald Élie ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Nash Equilibrium ◽  
Population Distribution ◽  
Mean Field ◽  
High Dimensional ◽  
Mean Field Games ◽  
Fictitious Play ◽  
Mean Field Game ◽  
Best Response

We present a method enabling a large number of agents to learn how to flock. This problem has drawn a lot of interest but requires many structural assumptions and is tractable only in small dimensions. We phrase this problem as a Mean Field Game (MFG), where each individual chooses its own acceleration depending on the population behavior. Combining Deep Reinforcement Learning (RL) and Normalizing Flows (NF), we obtain a tractable solution requiring only very weak assumptions. Our algorithm finds a Nash Equilibrium and the agents adapt their velocity to match the neighboring flock’s average one. We use Fictitious Play and alternate: (1) computing an approximate best response with Deep RL, and (2) estimating the next population distribution with NF. We show numerically that our algorithm can learn multi-group or high-dimensional flocking with obstacles.

scholarly journals Learning in mean field games: The fictitious play

2017 ◽  
Vol 23 (2) ◽  
pp. 569-591 ◽  
Author(s):  
Pierre Cardaliaguet ◽  
Saeed Hadikhanloo
Keyword(s):  
Differential Games ◽  
Mean Field ◽  
Mean Field Games ◽  
Fictitious Play ◽  
Mean Field Game ◽  
Interacting Agents ◽  
Learning Procedure ◽  
Equilibrium Configurations ◽  
The Mean

Mean Field Game systems describe equilibrium configurations in differential games with infinitely many infinitesimal interacting agents. We introduce a learning procedure (similar to the Fictitious Play) for these games and show its convergence when the Mean Field Game is potential.

scholarly journals A Probabilistic Approach to Extended Finite State Mean Field Games

2021 ◽  
Author(s):  
René Carmona ◽  
Peiqi Wang
Keyword(s):  
Nash Equilibrium ◽  
Continuous Time ◽  
Probabilistic Approach ◽  
Mean Field ◽  
Mean Field Games ◽  
Weak Formulation ◽  
Mean Field Game ◽  
Alternative Description ◽  
Finite State ◽  
The Mean

We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chains by means of semimartingales and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control at the same time, but also extends the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game as well as how the equilibrium of a mean field game consists of an approximative Nash equilibrium for the game with a finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states.

scholarly journals Quantum Mean-Field Games with the Observations of Counting Type

Games ◽  
2021 ◽  
Vol 12 (1) ◽  
pp. 7
Author(s):  
Vassili N. Kolokoltsov
Keyword(s):  
Game Theory ◽  
Nash Equilibrium ◽  
Open Problem ◽  
Existence And Uniqueness ◽  
Existence Of Solutions ◽  
Mean Field ◽  
Mean Field Games ◽  
Quantum Game ◽  
Quantum Games ◽  
Interesting Open Problem

Quantum games and mean-field games (MFG) represent two important new branches of game theory. In a recent paper the author developed quantum MFGs merging these two branches. These quantum MFGs were based on the theory of continuous quantum observations and filtering of diffusive type. In the present paper we develop the analogous quantum MFG theory based on continuous quantum observations and filtering of counting type. However, proving existence and uniqueness of the solutions for resulting limiting forward-backward system based on jump-type processes on manifolds seems to be more complicated than for diffusions. In this paper we only prove that if a solution exists, then it gives an ϵ-Nash equilibrium for the corresponding N-player quantum game. The existence of solutions is suggested as an interesting open problem.

scholarly journals Price Formation and Optimal Trading in Intraday Electricity Markets with a Major Player

Risks ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 133
Author(s):  
Olivier Féron ◽  
Peter Tankov ◽  
Laura Tinsi
Keyword(s):  
Nash Equilibrium ◽  
Electricity Markets ◽  
Mean Field ◽  
Optimal Strategies ◽  
Price Formation ◽  
Mean Field Games ◽  
Optimal Trading ◽  
Major Player ◽  
Major Producer ◽  
Small Producers

We study price formation in intraday electricity markets in the presence of intermittent renewable generation. We consider the setting where a major producer may interact strategically with a large number of small producers. Using stochastic control theory, we identify the optimal strategies of agents with market impact and exhibit the Nash equilibrium in a closed form in the asymptotic framework of mean field games with a major player.

scholarly journals Viability analysis of the first-order mean field games

2020 ◽  
Vol 26 ◽  
pp. 33
Author(s):  
Yurii Averboukh
Keyword(s):  
Mean Field ◽  
Initial Distribution ◽  
Initial Time ◽  
Mean Field Games ◽  
Sufficient Condition ◽  
Mean Field Game ◽  
First Order ◽  
Viability Analysis ◽  
A Value ◽  
Object Of Study

In the paper, we examine the dependence of the solution of the deterministic mean field game on the initial distribution of players. The main object of study is the mapping which assigns to the initial time and the initial distribution of players the set of expected rewards of the representative player corresponding to solutions of mean field game. This mapping can be regarded as a value multifunction. We obtain the sufficient condition for a multifunction to be a value multifunction. It states that if a multifunction is viable with respect to the dynamics generated by the original mean field game, then it is a value multifunction. Furthermore, the infinitesimal variant of this condition is derived.

scholarly journals Schrödinger approach to Mean Field Games with negative coordination

2020 ◽  
Vol 9 (4) ◽  
Author(s):  
Thibault Bonnemain ◽  
Thierry Gobron ◽  
Denis Ullmo
Keyword(s):  
Mean Field ◽  
Time Limit ◽  
External Potential ◽  
Mean Field Games ◽  
Relative Importance ◽  
Mean Field Game ◽  
Non Linear ◽  
The Mean ◽  
The Way

Mean Field Games provide a powerful framework to analyze the dynamics of a large number of controlled agents in interaction. Here we consider such systems when the interactions between agents result in a negative coordination and analyze the behavior of the associated system of coupled PDEs using the now well established correspondence with the non linear Schrödinger equation. We focus on the long optimization time limit and on configurations such that the game we consider goes through different regimes in which the relative importance of disorder, interactions between agents and external potential vary, which makes possible to get insights on the role of the forward-backward structure of the Mean Field Game equations in relation with the way these various regimes are connected.

scholarly journals A Novel Mean Field Game-Based Strategy for Charging Electric Vehicles in Solar Powered Parking Lots

Energies ◽  
2021 ◽  
Vol 14 (24) ◽  
pp. 8517
Author(s):  
Samuel M. Muhindo ◽  
Roland P. Malhamé ◽  
Geza Joos
Keyword(s):  
Nash Equilibrium ◽  
Solar Energy ◽  
Electric Vehicles ◽  
Large Population ◽  
Mean Field ◽  
Mean Field Games ◽  
Parking Lot ◽  
Equal Sharing ◽  
Solar Powered ◽  
Energy Forecast

We develop a strategy, with concepts from Mean Field Games (MFG), to coordinate the charging of a large population of battery electric vehicles (BEVs) in a parking lot powered by solar energy and managed by an aggregator. A yearly parking fee is charged for each BEV irrespective of the amount of energy extracted. The goal is to share the energy available so as to minimize the standard deviation (STD) of the state of charge (SOC) of batteries when the BEVs are leaving the parking lot, while maintaining some fairness and decentralization criteria. The MFG charging laws correspond to the Nash equilibrium induced by quadratic cost functions based on an inverse Nash equilibrium concept and designed to favor the batteries with the lower SOCs upon arrival. While the MFG charging laws are strictly decentralized, they guarantee that a mean of instantaneous charging powers to the BEVs follows a trajectory based on the solar energy forecast for the day. That day ahead forecast is broadcasted to the BEVs which then gauge the necessary SOC upon leaving their home. We illustrate the advantages of the MFG strategy for the case of a typical sunny day and a typical cloudy day when compared to more straightforward strategies: first come first full/serve and equal sharing. The behavior of the charging strategies is contrasted under conditions of random arrivals and random departures of the BEVs in the parking lot.

scholarly journals A machine learning framework for solving high-dimensional mean field game and mean field control problems

2020 ◽  
Vol 117 (17) ◽  
pp. 9183-9193
Author(s):  
Lars Ruthotto ◽  
Stanley J. Osher ◽  
Wuchen Li ◽  
Levon Nurbekyan ◽  
Samy Wu Fung
Keyword(s):  
Machine Learning ◽  
Numerical Methods ◽  
Mean Field ◽  
Two Dimensions ◽  
High Dimensional ◽  
Mean Field Games ◽  
Spatial Discretization ◽  
Learning Framework ◽  
Field Control ◽  
Crowd Motion

Mean field games (MFG) and mean field control (MFC) are critical classes of multiagent models for the efficient analysis of massive populations of interacting agents. Their areas of application span topics in economics, finance, game theory, industrial engineering, crowd motion, and more. In this paper, we provide a flexible machine learning framework for the numerical solution of potential MFG and MFC models. State-of-the-art numerical methods for solving such problems utilize spatial discretization that leads to a curse of dimensionality. We approximately solve high-dimensional problems by combining Lagrangian and Eulerian viewpoints and leveraging recent advances from machine learning. More precisely, we work with a Lagrangian formulation of the problem and enforce the underlying Hamilton–Jacobi–Bellman (HJB) equation that is derived from the Eulerian formulation. Finally, a tailored neural network parameterization of the MFG/MFC solution helps us avoid any spatial discretization. Our numerical results include the approximate solution of 100-dimensional instances of optimal transport and crowd motion problems on a standard work station and a validation using a Eulerian solver in two dimensions. These results open the door to much-anticipated applications of MFG and MFC models that are beyond reach with existing numerical methods.

Sign in / Sign up

Export Citation Format

Share Document