scholarly journals Applications of DEC-MDPs in Multi-Robot Systems

2012 ◽  
pp. 361-384
Author(s):  
Aurélie Beynier ◽  
Abdel-Illah Mouaddib
Keyword(s):  
Markov Decision Processes ◽  
Decentralized Control ◽  
Real World ◽  
Decision Processes ◽  
Decision Problems ◽  
Stochastic Environments ◽  
Robot Systems ◽  
Markov Decision ◽  
Multi Robot

In this chapter, we introduce problematics related to the decentralized control of multi-robot systems. We first describe some applicative domains and review the main characteristics of the decision problems the robots must deal with. Then, we review some existing approaches to solve problems of multiagent decentralized control in stochastic environments. We present the Decentralized Markov Decision Processes and discuss their applicability to real-world multi-robot applications. Then, we introduce OC-DEC-MDPs and 2V-DEC-MDPs which have been developed to increase the applicability of DEC-MDPs.

scholarly journals Applications of DEC-MDPs in Multi-Robot Systems

Robotics ◽  
2013 ◽  
pp. 143-165
Author(s):  
Aurélie Beynier ◽  
Abdel-Illah Mouaddib
Keyword(s):  
Decentralized Control ◽  
Decision Processes ◽  
Decision Problems ◽  
Large Set ◽  
Complex Environments ◽  
Uncertain Environments ◽  
Robot Systems ◽  
Markov Decision ◽  
Partially Observable ◽  
Multi Robot

Optimizing the operation of cooperative multi-robot systems that can cooperatively act in large and complex environments has become an important focal area of research. This issue is motivated by many applications involving a set of cooperative robots that have to decide in a decentralized way how to execute a large set of tasks in partially observable and uncertain environments. Such decision problems are encountered while developing exploration rovers, teams of patrolling robots, rescue-robot colonies, mine-clearance robots, et cetera. In this chapter, we introduce problematics related to the decentralized control of multi-robot systems. We first describe some applicative domains and review the main characteristics of the decision problems the robots must deal with. Then, we review some existing approaches to solve problems of multiagent decentralized control in stochastic environments. We present the Decentralized Markov Decision Processes and discuss their applicability to real-world multi-robot applications. Then, we introduce OC-DEC-MDPs and 2V-DEC-MDPs which have been developed to increase the applicability of DEC-MDPs.

scholarly journals Configurable Environments in Reinforcement Learning: An Overview

2022 ◽  
pp. 101-113
Author(s):  
Alberto Maria Metelli
Keyword(s):  
Reinforcement Learning ◽  
Markov Decision Processes ◽  
Learning Process ◽  
Real World ◽  
Decision Processes ◽  
Learning Problem ◽  
Complex Control ◽  
Control Frequency ◽  
Markov Decision ◽  
And Control

AbstractReinforcement Learning (RL) has emerged as an effective approach to address a variety of complex control tasks. In a typical RL problem, an agent interacts with the environment by perceiving observations and performing actions, with the ultimate goal of maximizing the cumulative reward. In the traditional formulation, the environment is assumed to be a fixed entity that cannot be externally controlled. However, there exist several real-world scenarios in which the environment offers the opportunity to configure some of its parameters, with diverse effects on the agent’s learning process. In this contribution, we provide an overview of the main aspects of environment configurability. We start by introducing the formalism of the Configurable Markov Decision Processes (Conf-MDPs) and we illustrate the solutions concepts. Then, we revise the algorithms for solving the learning problem in Conf-MDPs. Finally, we present two applications of Conf-MDPs: policy space identification and control frequency adaptation.

Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions

2017 ◽  
Vol 36 (2) ◽  
pp. 231-258 ◽  
Author(s):  
Shayegan Omidshafiei ◽  
Ali–Akbar Agha–Mohammadi ◽  
Christopher Amato ◽  
Shih–Yuan Liu ◽  
Jonathan P How ◽  
...  
Keyword(s):  
Markov Decision Processes ◽  
Large Scale ◽  
Decision Processes ◽  
Delivery Problem ◽  
Package Delivery ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
High Level ◽  
Partially Observable ◽  
Multi Robot

This work focuses on solving general multi-robot planning problems in continuous spaces with partial observability given a high-level domain description. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems. However, representing and solving Dec-POMDPs is often intractable for large problems. This work extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP) to take advantage of the high-level representations that are natural for multi-robot problems and to facilitate scalable solutions to large discrete and continuous problems. The Dec-POSMDP formulation uses task macro-actions created from lower-level local actions that allow for asynchronous decision-making by the robots, which is crucial in multi-robot domains. This transformation from Dec-POMDPs to Dec-POSMDPs with a finite set of automatically-generated macro-actions allows use of efficient discrete-space search algorithms to solve them. The paper presents algorithms for solving Dec-POSMDPs, which are more scalable than previous methods since they can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed algorithms are then evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent realistic problems and provide high-quality solutions for large-scale problems.

The Complexity of Decentralized Control of Markov Decision Processes

2002 ◽  
Vol 27 (4) ◽  
pp. 819-840 ◽  
Author(s):  
Daniel S. Bernstein ◽  
Robert Givan ◽  
Neil Immerman ◽  
Shlomo Zilberstein
Keyword(s):  
Markov Decision Processes ◽  
Decentralized Control ◽  
Decision Processes ◽  
Markov Decision
Sign in / Sign up

Export Citation Format

Share Document