Resonating Minds—Emergent Collaboration Through Hierarchical Active Inference

Working together on complex collaborative tasks requires agents to coordinate their actions. Doing this explicitly or completely prior to the actual interaction is not always possible nor sufficient. Agents also need to continuously understand the current actions of others and quickly adapt their own behavior appropriately. Here we investigate how efficient, automatic coordination processes at the level of mental states (intentions, goals), which we call belief resonance, can lead to collaborative situated problem-solving. We present a model of hierarchical active inference for collaborative agents (HAICA). It combines efficient Bayesian Theory of Mind processes with a perception–action system based on predictive processing and active inference. Belief resonance is realized by letting the inferred mental states of one agent influence another agent’s predictive beliefs about its own goals and intentions. This way, the inferred mental states influence the agent’s own task behavior without explicit collaborative reasoning. We implement and evaluate this model in the Overcooked domain, in which two agents with varying degrees of belief resonance team up to fulfill meal orders. Our results demonstrate that agents based on HAICA achieve a team performance comparable to recent state-of-the-art approaches, while incurring much lower computational costs. We also show that belief resonance is especially beneficial in settings where the agents have asymmetric knowledge about the environment. The results indicate that belief resonance and active inference allow for quick and efficient agent coordination and thus can serve as a building block for collaborative cognitive agents.

Leader-Following Multi-Agent Coordination Control Accompanied With Hierarchical Q(λ)-Learning for Pursuit

In this paper, we investigate a pursuit problem with multi-pursuer and single evader in a two-dimensional grid space with obstacles. Taking a different approach to previous studies, this paper aims to address a pursuit problem in which only some pursuers can directly access the evader’s position. It also proposes using a hierarchical Q(λ)-learning with improved reward, with simulation results indicating that the proposed method outperforms Q-learning.

Research on the coordination mechanism of major industrial project engineering and construction multi-agents based on structural holes theory

Industry is an important pillar of the national economy. Industrial projects are the most complex and difficult projects to control in the construction industry, and major industrial projects are even more complex and difficult to control. Multi-agent coordination is one of the core issues of industrial projects. Based on an analysis of the engineering and construction chains and agent relationships and agent networks of industrial projects, a complex network of the engineering and construction agents of industrial projects is established, and the complex network structural holes theory is applied to study the nonrepeated relationships among agents in industrial projects. Assuming agents are linked through contract relations and the most critical contract index between the agents in the contract amount, through structural hole analysis considering the EPC and PMC model, the aggregate constraint list is obtained, 2D network diagram and 3D network diagram are shown. According to the aggregate constraint value, the EPC contractor with the minimum aggregate constraint value and the project management company with the minimum aggregate constraint value are the critical agent in EPC and PMC model. By analyzing the complex network comprising different models of industrial projects, it is concluded that the characteristics of the agent maintain an advantage in competition, the coordination mechanism of the integration of agent interests, and multi-agent relations are considered to solve the multi-agent coordination problem in major industrial projects.

Neural Regret-Matching for Distributed Constraint Optimization Problems

Distributed constraint optimization problems (DCOPs) are a powerful model for multi-agent coordination and optimization, where information and controls are distributed among multiple agents by nature. Sampling-based algorithms are important incomplete techniques for solving medium-scale DCOPs. However, they use tables to exactly store all the information (e.g., costs, confidence bounds) to facilitate sampling, which limits their scalability. This paper tackles the limitation by incorporating deep neural networks in solving DCOPs for the first time and presents a neural-based sampling scheme built upon regret-matching. In the algorithm, each agent trains a neural network to approximate the regret related to its local problem and performs sampling according to the estimated regret. Furthermore, to ensure exploration we propose a regret rounding scheme that rounds small regret values to positive numbers. We theoretically show the regret bound of our algorithm and extensive evaluations indicate that our algorithm can scale up to large-scale DCOPs and significantly outperform the state-of-the-art methods.

Improving Multi-agent Coordination by Learning to Estimate Contention

We present a multi-agent learning algorithm, ALMA-Learning, for efficient and fair allocations in large-scale systems. We circumvent the traditional pitfalls of multi-agent learning (e.g., the moving target problem, the curse of dimensionality, or the need for mutually consistent actions) by relying on the ALMA heuristic as a coordination mechanism for each stage game. ALMA-Learning is decentralized, observes only own action/reward pairs, requires no inter-agent communication, and achieves near-optimal (<5% loss) and fair coordination in a variety of synthetic scenarios and a real-world meeting scheduling problem. The lightweight nature and fast learning constitute ALMA-Learning ideal for on-device deployment.

ORNInA: A decentralized, auction-based multi-agent coordination in ODT systems

On-Demand Transport (ODT) systems have attracted increasing attention in recent years. Traditional centralized dispatching can achieve optimal solutions, but NP-Hard complexity makes it unsuitable for online and dynamic problems. Centralized and decentralized heuristics can achieve fast, feasible solution at run-time with no guarantee on the quality. Starting from a feasible not optimal solution, we present in this paper a new solution model (ORNInA) consisting of two parallel coordination processes. The first one is a decentralized insertion-heuristic based algorithm to build vehicle schedules in order to solve a particular case of the dynamic Dial-A-Ride-Problem (DARP) as an ODT system, in which vehicles communicate via Vehicle-to-vehicle communication (V2V) and make decentralized decisions. The second coordination scheme is a continuous optimization process namely Pull-demand protocol, based on combinatorial auctions, in order to improve the quality of the global solution achieved by decentralized decision at run-time by exchanging resources between vehicles (k-opt). In its simplest implementation, k is set to 1 so that vehicles can exchange only one resource at a time. We evaluate and analyze the promising results of our contributed techniques on synthetic data for taxis operating in Saint-Étienne city, against a classical decentralized greedy approach and a centralized one that uses a classical mixed-integer linear program (MILP) solver.