scholarly journals Cemracs 2017: numerical probabilistic approach to MFG

2019 ◽  
Vol 65 ◽  
pp. 84-113
Author(s):  
Andrea Angiuli ◽  
Christy V. Graves ◽  
Houzhi Li ◽  
Jean-François Chassagneux ◽  
François Delarue ◽  
...  
Keyword(s):  
Optimization Problems ◽  
Continuation Method ◽  
Large Population ◽  
Probabilistic Approach ◽  
Mean Field ◽  
Picard Iteration ◽  
Mean Field Games ◽  
Benchmark Problems ◽  
Control Problems ◽  
Great Success

This project investigates numerical methods for solving fully coupled forward-backward stochastic differential equations (FBSDEs) of McKean-Vlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the potential application of these equations to optimization problems over a large population, say for instance mean field games (MFG) and optimal mean field control problems. Theory for this kind of problems has met with great success since the early works on mean field games by Lasry and Lions, see [29], and by Huang, Caines, and Malhamé, see [26]. Generally speaking, the purpose is to understand the continuum limit of optimizers or of equilibria (say in Nash sense) as the number of underlying players tends to infinity. When approached from the probabilistic viewpoint, solutions to these control problems (or games) can be described by coupled mean field FBSDEs, meaning that the coefficients depend upon the own marginal laws of the solution. In this note, we detail two methods for solving such FBSDEs which we implement and apply to five benchmark problems. The first method uses a tree structure to represent the pathwise laws of the solution, whereas the second method uses a grid discretization to represent the time marginal laws of the solutions. Both are based on a Picard scheme; importantly, we combine each of them with a generic continuation method that permits to extend the time horizon (or equivalently the coupling strength between the two equations) for which the Picard iteration converges.

scholarly journals Mean field games models of segregation

2017 ◽  
Vol 27 (01) ◽  
pp. 75-113 ◽  
Author(s):  
Yves Achdou ◽  
Martino Bardi ◽  
Marco Cirant
Keyword(s):  
Numerical Methods ◽  
Existence Of Solutions ◽  
Large Population ◽  
Mean Field ◽  
Mean Field Games ◽  
Residential Choice ◽  
Urban Settlements ◽  
Thomas Schelling ◽  
Two Populations ◽  
Large Population Limit

This paper introduces and analyzes some models in the framework of mean field games (MFGs) describing interactions between two populations motivated by the studies on urban settlements and residential choice by Thomas Schelling. For static games, a large population limit is proved. For differential games with noise, the existence of solutions is established for the systems of partial differential equations of MFG theory, in the stationary and in the evolutive case. Numerical methods are proposed with several simulations. In the examples and in the numerical results, particular emphasis is put on the phenomenon of segregation between the populations.

scholarly journals A Novel Mean Field Game-Based Strategy for Charging Electric Vehicles in Solar Powered Parking Lots

Energies ◽  
2021 ◽  
Vol 14 (24) ◽  
pp. 8517
Author(s):  
Samuel M. Muhindo ◽  
Roland P. Malhamé ◽  
Geza Joos
Keyword(s):  
Nash Equilibrium ◽  
Solar Energy ◽  
Electric Vehicles ◽  
Large Population ◽  
Mean Field ◽  
Mean Field Games ◽  
Parking Lot ◽  
Equal Sharing ◽  
Solar Powered ◽  
Energy Forecast

We develop a strategy, with concepts from Mean Field Games (MFG), to coordinate the charging of a large population of battery electric vehicles (BEVs) in a parking lot powered by solar energy and managed by an aggregator. A yearly parking fee is charged for each BEV irrespective of the amount of energy extracted. The goal is to share the energy available so as to minimize the standard deviation (STD) of the state of charge (SOC) of batteries when the BEVs are leaving the parking lot, while maintaining some fairness and decentralization criteria. The MFG charging laws correspond to the Nash equilibrium induced by quadratic cost functions based on an inverse Nash equilibrium concept and designed to favor the batteries with the lower SOCs upon arrival. While the MFG charging laws are strictly decentralized, they guarantee that a mean of instantaneous charging powers to the BEVs follows a trajectory based on the solar energy forecast for the day. That day ahead forecast is broadcasted to the BEVs which then gauge the necessary SOC upon leaving their home. We illustrate the advantages of the MFG strategy for the case of a typical sunny day and a typical cloudy day when compared to more straightforward strategies: first come first full/serve and equal sharing. The behavior of the charging strategies is contrasted under conditions of random arrivals and random departures of the BEVs in the parking lot.

scholarly journals The Pareto Tracer for General Inequality Constrained Multi-Objective Optimization Problems

2020 ◽  
Vol 25 (4) ◽  
pp. 80
Author(s):  
Fernanda Beltrán ◽  
Oliver Cuate ◽  
Oliver Schütze
Keyword(s):  
Optimization Problems ◽  
Continuation Method ◽  
Optimization Methods ◽  
New Method ◽  
Continuation Methods ◽  
Benchmark Problems ◽  
Multi Objective Optimization ◽  
Financial Applications ◽  
General Inequality ◽  
Multi Objective

Problems where several incommensurable objectives have to be optimized concurrently arise in many engineering and financial applications. Continuation methods for the treatment of such multi-objective optimization methods (MOPs) are very efficient if all objectives are continuous since in that case one can expect that the solution set forms at least locally a manifold. Recently, the Pareto Tracer (PT) has been proposed, which is such a multi-objective continuation method. While the method works reliably for MOPs with box and equality constraints, no strategy has been proposed yet to adequately treat general inequalities, which we address in this work. We formulate the extension of the PT and present numerical results on some selected benchmark problems. The results indicate that the new method can indeed handle general MOPs, which greatly enhances its applicability.

scholarly journals On the Convergence of Model Free Learning in Mean Field Games

2020 ◽  
Vol 34 (05) ◽  
pp. 7143-7150
Author(s):  
Romuald Elie ◽  
Julien Pérolat ◽  
Mathieu Laurière ◽  
Matthieu Geist ◽  
Olivier Pietquin
Keyword(s):  
Learning Algorithms ◽  
Large Population ◽  
Single Agent ◽  
Mean Field ◽  
Space Environment ◽  
Iteration Step ◽  
Mean Field Games ◽  
Scalable Algorithms ◽  
Model Free ◽  
Approximate Nash Equilibrium

Learning by experience in Multi-Agent Systems (MAS) is a difficult and exciting task, due to the lack of stationarity of the environment, whose dynamics evolves as the population learns. In order to design scalable algorithms for systems with a large population of interacting agents (e.g., swarms), this paper focuses on Mean Field MAS, where the number of agents is asymptotically infinite. Recently, a very active burgeoning field studies the effects of diverse reinforcement learning algorithms for agents with no prior information on a stationary Mean Field Game (MFG) and learn their policy through repeated experience. We adopt a high perspective on this problem and analyze in full generality the convergence of a fictitious iterative scheme using any single agent learning algorithm at each step. We quantify the quality of the computed approximate Nash equilibrium, in terms of the accumulated errors arising at each learning iteration step. Notably, we show for the first time convergence of model free learning algorithms towards non-stationary MFG equilibria, relying only on classical assumptions on the MFG dynamics. We illustrate our theoretical results with a numerical experiment in a continuous action-space environment, where the approximate best response of the iterative fictitious play scheme is computed with a deep RL algorithm.

scholarly journals Probabilistic Approach to Finite State Mean Field Games

2018 ◽  
Vol 81 (2) ◽  
pp. 253-300 ◽  
Author(s):  
Alekos Cecchin ◽  
Markus Fischer
Keyword(s):  
Probabilistic Approach ◽  
Mean Field ◽  
Mean Field Games ◽  
Finite State

scholarly journals A Probabilistic Approach to Extended Finite State Mean Field Games

2021 ◽  
Author(s):  
René Carmona ◽  
Peiqi Wang
Keyword(s):  
Nash Equilibrium ◽  
Continuous Time ◽  
Probabilistic Approach ◽  
Mean Field ◽  
Mean Field Games ◽  
Weak Formulation ◽  
Mean Field Game ◽  
Alternative Description ◽  
Finite State ◽  
The Mean

We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chains by means of semimartingales and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control at the same time, but also extends the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game as well as how the equilibrium of a mean field game consists of an approximative Nash equilibrium for the game with a finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states.

Sign in / Sign up

Export Citation Format

Share Document