Multi-robot Cooperation Strategy in a Partially Observable Markov Game Using Enhanced Deep Deterministic Policy Gradient

2019 ◽  
pp. 3-10
Author(s):  
Qirong Tang ◽  
Jingtao Zhang ◽  
Fangchao Yu ◽  
Pengjie Xu ◽  
Zhongqun Zhang
Keyword(s):  
Markov Game ◽  
Policy Gradient ◽  
Cooperation Strategy ◽  
Partially Observable Markov ◽  
Robot Cooperation ◽  
Partially Observable ◽  
Multi Robot

Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions

2017 ◽  
Vol 36 (2) ◽  
pp. 231-258 ◽  
Author(s):  
Shayegan Omidshafiei ◽  
Ali–Akbar Agha–Mohammadi ◽  
Christopher Amato ◽  
Shih–Yuan Liu ◽  
Jonathan P How ◽  
...  
Keyword(s):  
Markov Decision Processes ◽  
Large Scale ◽  
Decision Processes ◽  
Delivery Problem ◽  
Package Delivery ◽  
Markov Decision ◽  
Partially Observable Markov ◽  
High Level ◽  
Partially Observable ◽  
Multi Robot

This work focuses on solving general multi-robot planning problems in continuous spaces with partial observability given a high-level domain description. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems. However, representing and solving Dec-POMDPs is often intractable for large problems. This work extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP) to take advantage of the high-level representations that are natural for multi-robot problems and to facilitate scalable solutions to large discrete and continuous problems. The Dec-POSMDP formulation uses task macro-actions created from lower-level local actions that allow for asynchronous decision-making by the robots, which is crucial in multi-robot domains. This transformation from Dec-POMDPs to Dec-POSMDPs with a finite set of automatically-generated macro-actions allows use of efficient discrete-space search algorithms to solve them. The paper presents algorithms for solving Dec-POSMDPs, which are more scalable than previous methods since they can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed algorithms are then evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent realistic problems and provide high-quality solutions for large-scale problems.

Comparing Plume-Tracing Algorithms in Obstructed Multi-Source Environments

2013 ◽  
Vol 756-759 ◽  
pp. 228-232
Author(s):  
Yu Li Zhang ◽  
Xiao Ping Ma
Keyword(s):  
Search Performance ◽  
Robot System ◽  
System A ◽  
Plume Tracing ◽  
Environment Simulation ◽  
Cooperation Strategy ◽  
The Common ◽  
Robot Cooperation ◽  
Dynamics Simulations ◽  
Multi Robot

In this paper, we compare the common plume-tracing algorithms: chemotaxis and anemotaxis in obstructed multi-source environment using multi-robot system. A multi-robot cooperation strategy with virtual physics force, which includes structure formation force, goal force, obstacle avoidance force, repulsion force and rotary force, is proposed. First, plume model with two sources in three obstacles environment is constructed by computation fluid dynamics simulations. Second, parallel searches by two groups robots with chemotaxis and anemotaxis are used to locate two sources in obstructed environment. Simulation comparison experiment with two plume-tracing algorithms discussed the influence of the varied wind direction/ speed frequency and methane release frequency and different initial positions of two groups to the search performance. Finally, the comparative result is illustrated.

A PSO-Based Approach with Fuzzy Obstacle Avoidance for Cooperative Multi-Robots in Unknown Environments

2016 ◽  
Vol 15 (01) ◽  
pp. 1650001 ◽  
Author(s):  
Yifan Cai ◽  
Simon X. Yang
Keyword(s):  
Obstacle Avoidance ◽  
Fitness Function ◽  
Field Function ◽  
Robot Trajectory ◽  
Unknown Environments ◽  
Cooperation Strategy ◽  
Key Issues ◽  
Robot Cooperation ◽  
Target Searching ◽  
Multi Robot

Cooperative exploration in unknown environments is fundamentally important in robotics, where the real-time path planning and proper task allocation strategies are the key issues for multi-robot cooperation. In this paper, a PSO-based approach, combined with a fuzzy obstacle avoidance module, is proposed for cooperative robots to accomplish target searching and foraging tasks in unknown environments. The proposed cooperation strategy for a multi-robot system makes use of the potential field function as the fitness function of PSO, while the proposed fuzzy obstacle-avoidance module improves the smoothness of robot trajectory. In the simulation studies, several scenarios with and without the fuzzy module are investigated. The robot trajectory smoothness improvement is demonstrated through the comparative studies.

Localizing Multiple Odor Sources Using Virtual Physics Based Robots

2013 ◽  
Vol 756-759 ◽  
pp. 223-227
Author(s):  
Yu Li Zhang ◽  
Xiao Ping Ma
Keyword(s):  
Search Performance ◽  
Simulation Environment ◽  
Robot System ◽  
Comparative Result ◽  
System A ◽  
Cooperation Strategy ◽  
Robot Cooperation ◽  
Dynamics Simulations ◽  
Multiple Chemical ◽  
Multi Robot

This paper is concerned with the problem of multiple chemical sources localization using multi-robot system. A multi-robot cooperation strategy with virtual physics force, which includes structure formation force, goal force, repulsion force and rotary force, is proposed. First, in order to test the effectiveness of the proposed strategy, two sources plume model are constructed by computation fluid dynamics simulations. Second, parallel search by two groups robots is used to locate two sources in simulation environment. With the purpose of preventing two groups from locating the same source, we proposed a rotary force which made each subgroup can locate different chemical source. Simulation experiment discussed the influence of the varied wind direction/ speed frequency and methane release frequency and different initial positions of two groups to the search performance. Finally, the comparative result about them is illustrated.

Sign in / Sign up

Export Citation Format

Share Document