A Novel Mean-Field-Game-Type Optimal Control for Very Large-Scale Multiagent Systems

This paper considers a mean field game model inspired by crowd motion where agents want to leave a given bounded domain through a part of its boundary in minimal time. Each agent is free to move in any direction, but their maximal speed is bounded in terms of the average density of agents around their position in order to take into account congestion phenomena. After a preliminary study of the corresponding minimal-time optimal control problem, we formulate the mean field game in a Lagrangian setting and prove existence of Lagrangian equilibria using a fixed point strategy. We provide a further study of equilibria under the assumption that agents may leave the domain through the whole boundary, in which case equilibria are described through a system of a continuity equation on the distribution of agents coupled with a Hamilton–Jacobi equation on the value function of the optimal control problem solved by each agent. This is possible thanks to the semiconcavity of the value function, which follows from some further regularity properties of optimal trajectories obtained through Pontryagin Maximum Principle. Simulations illustrate the behavior of equilibria in some particular situations.

Download Full-text

Decentralized Adaptive Optimal Control for Massive Multi-agent Systems Using Mean Field Game with Self-Organizing Neural Networks

2019 IEEE 58th Conference on Decision and Control (CDC) ◽

10.1109/cdc40024.2019.9029540 ◽

2019 ◽

Author(s):

Zejian Zhou ◽

Hao Xu

Keyword(s):

Neural Networks ◽

Optimal Control ◽

Mean Field ◽

Multi Agent Systems ◽

Adaptive Optimal Control ◽

Mean Field Game ◽

Agent Systems ◽

Multi Agent ◽

Self Organizing

Download Full-text

Stability Analysis in Mean-Field Games via an Evans Function Approach

Volume 3: Modeling and Validation; Multi-Agent and Networked Systems; Path Planning and Motion Control; Tracking Control Systems; Unmanned Aerial Vehicles (UAVs) and Application; Unmanned Ground and Aerial Vehicles; Vibration in Mechanical Systems; Vibrations and Control of Systems; Vibrations: Modeling, Analysis, and Control ◽

10.1115/dscc2018-8926 ◽

2018 ◽

Author(s):

Piyush Grover

Keyword(s):

Optimal Control ◽

Stability Analysis ◽

Mean Field ◽

Evans Function ◽

Function Approach ◽

Mean Field Games ◽

Mean Field Game ◽

Large Populations ◽

Multi Agent ◽

Hamilton Jacobi Bellman

This work is concerned with stability analysis of stationary and time-varying equilibria in a class of mean-field games that relate to multi-agent control problems of flocking and swarming. The mean-field game framework is a non-cooperative model of distributed optimal control in large populations, and characterizes the optimal control for a representative agent in Nash-equilibrium with the population. A mean-field game model is described by a coupled PDE system of forward-in-time Fokker-Planck (FP) equation for density of agents, and a backward-in-time Hamilton-Jacobi-Bellman (HJB) equation for control. The linear stability analysis of fixed points of these equations typically proceeds via numerical computation of spectrum of the linearized MFG operator. We explore the Evans function approach that provides a geometric alternative to solving the characteristic equation.

Download Full-text

Reinforcement Learning-based Decentralized Optimal Control for Large-Scale Multi-agent System by Using Neural Networks and Discrete-time Mean Field Games

10.1109/ijcnn52387.2021.9534288 ◽

2021 ◽

Author(s):

Zejian Zhou ◽

Yuzhu Zhang ◽

Hao Xu

Keyword(s):

Neural Networks ◽

Optimal Control ◽

Reinforcement Learning ◽

Discrete Time ◽

Large Scale ◽

Mean Field ◽

Mean Field Games ◽

Multi Agent System ◽

Agent System ◽

Multi Agent

Download Full-text

On a mean field game optimal control approach modeling fast exit scenarios in human crowds

52nd IEEE Conference on Decision and Control ◽

10.1109/cdc.2013.6760360 ◽

2013 ◽

Cited By ~ 5

Author(s):

Martin Burger ◽

Marco Di Francesco ◽

Peter A. Markowich ◽

Marie-Therese Wolfram

Keyword(s):

Optimal Control ◽

Mean Field ◽

Mean Field Game ◽

Control Approach ◽

Optimal Control Approach

Download Full-text

On closed-loop equilibrium strategies for mean-field stochastic linear quadratic problems

ESAIM Control Optimisation and Calculus of Variations ◽

10.1051/cocv/2019057 ◽

2020 ◽

Vol 26 ◽

pp. 41

Author(s):

Tianxiao Wang

Keyword(s):

Optimal Control ◽

Closed Loop ◽

Optimal Control Problems ◽

Mean Field ◽

Open Loop ◽

Time Inconsistency ◽

Control Problems ◽

Linear Quadratic ◽

Linear Quadratic Optimal Control ◽

Equilibrium Strategies

This article is concerned with linear quadratic optimal control problems of mean-field stochastic differential equations (MF-SDE) with deterministic coefficients. To treat the time inconsistency of the optimal control problems, linear closed-loop equilibrium strategies are introduced and characterized by variational approach. Our developed methodology drops the delicate convergence procedures in Yong [Trans. Amer. Math. Soc. 369 (2017) 5467–5523]. When the MF-SDE reduces to SDE, our Riccati system coincides with the analogue in Yong [Trans. Amer. Math. Soc. 369 (2017) 5467–5523]. However, these two systems are in general different from each other due to the conditional mean-field terms in the MF-SDE. Eventually, the comparisons with pre-committed optimal strategies, open-loop equilibrium strategies are given in details.

Download Full-text