Hamiltonian coordination primitives for decentralized multiagent navigation

We focus on decentralized navigation among multiple non-communicating agents in continuous domains without explicit traffic rules, such as sidewalks, hallways, or squares. Following collision-free motion in such domains requires effective mechanisms of multiagent behavior prediction. Although this prediction problem can be shown to be NP-hard, humans are often capable of solving it efficiently by leveraging sophisticated mechanisms of implicit coordination. Inspired by the human paradigm, we propose a novel topological formalism that explicitly models multiagent coordination. Our formalism features both geometric and algebraic descriptions enabling the use of standard gradient-based optimization techniques for trajectory generation but also symbolic inference over coordination strategies. In this article, we contribute (a) HCP (Hamiltonian Coordination Primitives), a novel multiagent trajectory-generation pipeline that accommodates spatiotemporal constraints formulated as symbolic topological specifications corresponding to a desired coordination strategy; (b) HCPnav, an online planning framework for decentralized collision avoidance that generates motion by following multiagent trajectory primitives corresponding to high-likelihood, low-cost coordination strategies. Through a series of challenging trajectory-generation experiments, we show that HCP outperforms a trajectory-optimization baseline in generating trajectories of desired topological specifications in terms of success rate and computational efficiency. Finally, through a variety of navigation experiments, we illustrate the efficacy of HCPnav in handling challenging multiagent navigation scenarios under homogeneous or heterogeneous agents across a series of environments of different geometry.

Consensus, cooperative learning, and flocking for multiagent predator avoidance

Multiagent coordination is highly desirable with many uses in a variety of tasks. In nature, the phenomenon of coordinated flocking is highly common with applications related to defending or escaping from predators. In this article, a hybrid multiagent system that integrates consensus, cooperative learning, and flocking control to determine the direction of attacking predators and learns to flock away from them in a coordinated manner is proposed. This system is entirely distributed requiring only communication between neighboring agents. The fusion of consensus and collaborative reinforcement learning allows agents to cooperatively learn in a variety of multiagent coordination tasks, but this article focuses on flocking away from attacking predators. The results of the flocking show that the agents are able to effectively flock to a target without collision with each other or obstacles. Multiple reinforcement learning methods are evaluated for the task with cooperative learning utilizing function approximation for state-space reduction performing the best. The results of the proposed consensus algorithm show that it provides quick and accurate transmission of information between agents in the flock. Simulations are conducted to show and validate the proposed hybrid system in both one and two predator environments, resulting in an efficient cooperative learning behavior. In the future, the system of using consensus to determine the state and reinforcement learning to learn the states can be applied to additional multiagent tasks.