Enabling Artificial Intelligence Studies in Off-Road Mobility Through Physics-based Simulation of Multi-Agent Scenarios

2021 ◽  
Author(s):  
Aaron Young ◽  
Jay Taves ◽  
Asher Elmquist ◽  
Simone Benatti ◽  
Alessandro Tasora ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Message Passing ◽  
Message Passing Interface ◽  
Autonomous Agents ◽  
Time Space ◽  
Intelligence Studies ◽  
Multi Agent ◽  
Physics Based Simulation ◽  
Design And Testing

Abstract We describe a simulation environment that enables the design and testing of control policies for off-road mobility of autonomous agents. The environment is demonstrated in conjunction with the training and assessment of a reinforcement learning policy that uses sensor fusion and inter-agent communication to enable the movement of mixed convoys of human-driven and autonomous vehicles. Policies learned on rigid terrain are shown to transfer to hard (silt-like) and soft (snow-like) deformable terrains. The environment described performs the following: multi-vehicle multibody dynamics co-simulation in a time/space-coherent infrastructure that relies on the Message Passing Interface standard for low-latency parallel computing; sensor simulation (e.g., camera, GPU, IMU); simulation of a virtual world that can be altered by the agents present in the simulation; training that uses reinforcement learning to 'teach' the autonomous vehicles to drive in an obstacle-riddled course. The software stack described is open source.

Enabling Artificial Intelligence Studies in Off-Road Mobility Through Physics-Based Simulation of Multi-Agent Scenarios

2021 ◽  
Author(s):  
Aaron Young ◽  
Jay Taves ◽  
Asher Elmquist ◽  
Radu Serban ◽  
Dan Negrut ◽  
...  
Keyword(s):  
Autonomous Vehicles ◽  
Autonomous Agents ◽  
Simulation Environment ◽  
Agent Communication ◽  
Testing Environment ◽  
Simulation Engine ◽  
Intelligence Studies ◽  
Multi Agent ◽  
Physics Based Simulation ◽  
Sensor Modeling

Abstract We describe a simulation environment that enables the development and testing of control policies for off-road mobility of autonomous agents. The environment is demonstrated in conjunction with the design and assessment of a reinforcement learning policy that uses sensor fusion and inter-agent communication to enable the movement of mixed convoys of human-driven and autonomous vehicles. Policies are learned on rigid terrain and are subsequently shown to transfer successfully to hard (silt-like) and soft (snow-like) deformable terrains. The enabling simulation environment is developed from the high fidelity, physics-based simulation engine Chrono. Five Chrono modules are employed herein: Chrono::Engine, Chrono::Vehicle, PyChrono, SynChrono and Chrono::Sensor. Vehicle’s are modeled using Chrono::Engine and Chrono::Vehicle and deployed on deformable terrain within the training/testing environment. Utilizing the Python interface to the C++ Chrono API called PyChrono and OpenAI Gym’s supporting infrastructure, training is conducted in a GymChrono learning environment. The GymChrono-generated policy is subsequently deployed for testing in SynChrono, a scalable, cluster-deployable multi-agent testing infrastructure built on MPI. SynChrono facilitates inter-agent communication and maintains time and space coherence between agents. A sensor modeling tool, Chrono::Sensor, supplies sensing data that is used to inform agents during the learning and inference processes. The software stack and the Chrono simulator are both open source. Relevant movies: [1].

scholarly journals A Programming Approach to Collective Autonomy

2021 ◽  
Vol 10 (2) ◽  
pp. 27
Author(s):  
Roberto Casadei ◽  
Gianluca Aguzzi ◽  
Mirko Viroli
Keyword(s):  
Autonomous Vehicles ◽  
Autonomic Computing ◽  
Autonomous Agents ◽  
Programming Approach ◽  
Multi Agent Systems ◽  
Agent Systems ◽  
Multi Agent ◽  
Simulated Case ◽  
Complex Settings

Research and technology developments on autonomous agents and autonomic computing promote a vision of artificial systems that are able to resiliently manage themselves and autonomously deal with issues at runtime in dynamic environments. Indeed, autonomy can be leveraged to unburden humans from mundane tasks (cf. driving and autonomous vehicles), from the risk of operating in unknown or perilous environments (cf. rescue scenarios), or to support timely decision-making in complex settings (cf. data-centre operations). Beyond the results that individual autonomous agents can carry out, a further opportunity lies in the collaboration of multiple agents or robots. Emerging macro-paradigms provide an approach to programming whole collectives towards global goals. Aggregate computing is one such paradigm, formally grounded in a calculus of computational fields enabling functional composition of collective behaviours that could be proved, under certain technical conditions, to be self-stabilising. In this work, we address the concept of collective autonomy, i.e., the form of autonomy that applies at the level of a group of individuals. As a contribution, we define an agent control architecture for aggregate multi-agent systems, discuss how the aggregate computing framework relates to both individual and collective autonomy, and show how it can be used to program collective autonomous behaviour. We exemplify the concepts through a simulated case study, and outline a research roadmap towards reliable aggregate autonomy.

scholarly journals Multi-agent Reinforcement Learning for Autonomous Vehicles in Wireless Sensor Networks

2021 ◽  
Author(s):  
Haitham Afifi
Keyword(s):  
Wireless Sensor Networks ◽  
Reinforcement Learning ◽  
Sensor Networks ◽  
Real World ◽  
Autonomous Vehicles ◽  
Wireless Sensor ◽  
Acoustic Sensor Networks ◽  
Markov Decision ◽  
Multi Agent

<div>We develop a Deep Reinforcement Learning (DeepRL) based multi-agent algorithm to efficiently control</div><div>autonomous vehicles in the context of Wireless Sensor Networks (WSNs). In contrast to other applications, WSNs</div><div>have two metrics for performance evaluation. First, quality of information (QoI) which is used to measure the</div><div>quality of sensed data. Second, quality of service (QoS) which is used to measure the network’s performance. As</div><div>a use case, we consider wireless acoustic sensor networks; a group of speakers move inside a room and there</div><div>are microphones installed on vehicles for streaming the audio data. We formulate an appropriate Markov Decision</div><div>Process (MDP) and present, besides a centralized solution, a multi-agent Deep Q-learning solution to control the vehicles. We compare the proposed solutions to a naive heuristic and two different real-world implementations: microphones being hold or preinstalled. We show using simulations that the performance of autonomous vehicles in terms of QoI and QoS is better than the real-world implementation and the proposed heuristic. Additionally, we provide theoretical analysis of the performance with respect to WSNs dynamics, such as speed, rooms dimensions and speaker’s talking time.</div>

scholarly journals A hybrid heuristic parallel method of global optimization

2015 ◽  
pp. 242-255
Author(s):  
К.В. Пушкарев ◽  
В.Д. Кошур
Keyword(s):  
Global Optimization ◽  
Objective Function ◽  
Message Passing ◽  
Message Passing Interface ◽  
Optimization Problems ◽  
Parallel Method ◽  
Hybrid Heuristic ◽  
Generalized Regression Neural Networks ◽  
Multi Agent ◽  
Multiple Variables

Рассматривается задача нахождения глобального минимума непрерывной целевой функции многих переменных в области, имеющей вид многомерного параллелепипеда. Для решения сложных задач глобальной оптимизации предлагается гибридный эвристический параллельный метод глобальной оптимизации (ГЭПМ), основанный на комбинировании и гибридизации различных методов и технологии многоагентной системы. В состав ГЭПМ включены как новые методы (например, метод нейросетевой аппроксимации инверсных зависимостей, использующий обобщeнно-регрессионные нейронные сети (GRNN), отображающие значения целевой функции в значения координат), так и модифицированные классические методы (например, модифицированный метод Хука-Дживса). Кратко описывается программная реализация ГЭПМ в форме кроссплатформенной (на уровне исходного кода) программной библиотеки на языке C++, использующей обмен сообщениями через интерфейс MPI (Message Passing Interface). Приводятся результаты сравнения ГЭПМ с 21 современным методом глобальной оптимизации и генетическим алгоритмом на 28 тестовых целевых функциях 50 переменных. The problem of finding the global minimum of a continuous objective function of multiple variables in a multidimensional parallelepiped is considered. A hybrid heuristic parallel method for solving of complicated global optimization problems is proposed. The method is based on combining various methods and on the multi-agent technology. It consists of new methods (for example, the method of neural network approximation of inverse coordinate mappings that uses Generalized Regression Neural Networks (GRNN) to map the values of an objective function to coordinates) and modified classical methods (for example, the modified Hooke-Jeeves method). An implementation of the proposed method as a cross-platform (on the source code level) library written in the C++ language is briefly discussed. This implementation uses the message passing via MPI (Message Passing Interface). The method is compared with 21 modern methods of global optimization and with a genetic algorithm using 28 test objective functions of 50 variables.

scholarly journals Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning

2021 ◽  
Vol 35 (2) ◽  
Author(s):  
Jacopo Castellini ◽  
Frans A. Oliehoek ◽  
Rahul Savani ◽  
Shimon Whiteson
Keyword(s):  
Reinforcement Learning ◽  
Multiagent Systems ◽  
Autonomous Agents ◽  
Coordination Mechanism ◽  
Network Architectures ◽  
Value Functions ◽  
Multi Agent Systems ◽  
Learning Techniques ◽  
Coordination Requirements ◽  
Multi Agent

AbstractRecent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results extend those in Castellini et al. (Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS’19.International Foundation for Autonomous Agents and Multiagent Systems, pp 1862–1864, 2019) and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements.

scholarly journals Multi-agent Reinforcement Learning for Autonomous Vehicles in Wireless Sensor Networks

2021 ◽  
Author(s):  
Haitham Afifi
Keyword(s):  
Wireless Sensor Networks ◽  
Reinforcement Learning ◽  
Sensor Networks ◽  
Real World ◽  
Autonomous Vehicles ◽  
Wireless Sensor ◽  
Acoustic Sensor Networks ◽  
Markov Decision ◽  
Multi Agent

<div>We develop a Deep Reinforcement Learning (DeepRL) based multi-agent algorithm to efficiently control</div><div>autonomous vehicles in the context of Wireless Sensor Networks (WSNs). In contrast to other applications, WSNs</div><div>have two metrics for performance evaluation. First, quality of information (QoI) which is used to measure the</div><div>quality of sensed data. Second, quality of service (QoS) which is used to measure the network’s performance. As</div><div>a use case, we consider wireless acoustic sensor networks; a group of speakers move inside a room and there</div><div>are microphones installed on vehicles for streaming the audio data. We formulate an appropriate Markov Decision</div><div>Process (MDP) and present, besides a centralized solution, a multi-agent Deep Q-learning solution to control the vehicles. We compare the proposed solutions to a naive heuristic and two different real-world implementations: microphones being hold or preinstalled. We show using simulations that the performance of autonomous vehicles in terms of QoI and QoS is better than the real-world implementation and the proposed heuristic. Additionally, we provide theoretical analysis of the performance with respect to WSNs dynamics, such as speed, rooms dimensions and speaker’s talking time.</div>

Multi-level Parallelization of Genotype Imputation on Supercomputers

2020 ◽  
Vol 15 ◽  
Author(s):  
Weiwen Zhang ◽  
Long Wang ◽  
Theint Theint Aye ◽  
Juniarto Samsudin ◽  
Yongqing Zhu
Keyword(s):  
Association Study ◽  
Message Passing ◽  
High Performance ◽  
Message Passing Interface ◽  
Genome Wide Association Study ◽  
Job Scheduling ◽  
Genotype Imputation ◽  
Job Level ◽  
Multi Level ◽  
High Performance Requirement

Background: Genotype imputation as a service is developed to enable researchers to estimate genotypes on haplotyped data without performing whole genome sequencing. However, genotype imputation is computation intensive and thus it remains a challenge to satisfy the high performance requirement of genome wide association study (GWAS). Objective: In this paper, we propose a high performance computing solution for genotype imputation on supercomputers to enhance its execution performance. Method: We design and implement a multi-level parallelization that includes job level, process level and thread level parallelization, enabled by job scheduling management, message passing interface (MPI) and OpenMP, respectively. It involves job distribution, chunk partition and execution, parallelized iteration for imputation and data concatenation. Due to the design of multi-level parallelization, we can exploit the multi-machine/multi-core architecture to improve the performance of genotype imputation. Results: Experiment results show that our proposed method can outperform the Hadoop-based implementation of genotype imputation. Moreover, we conduct the experiments on supercomputers to evaluate the performance of the proposed method. The evaluation shows that it can significantly shorten the execution time, thus improving the performance for genotype imputation. Conclusion: The proposed multi-level parallelization, when deployed as an imputation as a service, will facilitate bioinformatics researchers in Singapore to conduct genotype imputation and enhance the association study.

Sign in / Sign up

Export Citation Format

Share Document