scholarly journals MADRaS : Multi Agent Driving Simulator

2021 ◽  
Vol 70 ◽  
pp. 1517-1555
Author(s):  
Anirban Santara ◽  
Sohan Rudra ◽  
Sree Aditya Buridi ◽  
Meha Kaushik ◽  
Abhishek Naik ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Motion Planning ◽  
Open Source ◽  
Real World ◽  
Driving Simulator ◽  
Autonomous Driving ◽  
Machine Learning Algorithms ◽  
Control Mode ◽  
Simulation Engine ◽  
Multi Agent

Autonomous driving has emerged as one of the most active areas of research as it has the promise of making transportation safer and more efficient than ever before. Most real-world autonomous driving pipelines perform perception, motion planning and action in a loop. In this work we present MADRaS, an open-source multi-agent driving simulator for use in the design and evaluation of motion planning algorithms for autonomous driving. Given a start and a goal state, the task of motion planning is to solve for a sequence of position, orientation and speed values in order to navigate between the states while adhering to safety constraints. These constraints often involve the behaviors of other agents in the environment. MADRaS provides a platform for constructing a wide variety of highway and track driving scenarios where multiple driving agents can be trained for motion planning tasks using reinforcement learning and other machine learning algorithms. MADRaS is built on TORCS, an open-source car-racing simulator. TORCS offers a variety of cars with different dynamic properties and driving tracks with different geometries and surface.  MADRaS inherits these functionalities from TORCS and introduces support for multi-agent training, inter-vehicular communication, noisy observations, stochastic actions, and custom traffic cars whose behaviors can be programmed to simulate challenging traffic conditions encountered in the real world. MADRaS can be used to create driving tasks whose complexities can be tuned along eight axes in well-defined steps. This makes it particularly suited for curriculum and continual learning. MADRaS is lightweight and it provides a convenient OpenAI Gym interface for independent control of each car. Apart from the primitive steering-acceleration-brake control mode of TORCS, MADRaS offers a hierarchical track-position – speed control mode that can potentially be used to achieve better generalization. MADRaS uses a UDP based client server model where the simulation engine is the server and each client is a driving agent. MADRaS uses multiprocessing to run each agent as a parallel process for efficiency and integrates well with popular reinforcement learning libraries like RLLib. We show experiments on single and multi-agent reinforcement learning with and without curriculum

scholarly journals Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving

2019 ◽  
Author(s):  
Akifumi Wachi
Keyword(s):  
Reinforcement Learning ◽  
Driving Simulator ◽  
Autonomous Driving ◽  
Development Phase ◽  
Rule Based ◽  
Safety Critical ◽  
Wide Range ◽  
Multi Agent ◽  
Failure Scenario

We examine the problem of adversarial reinforcement learning for multi-agent domains including a rule-based agent. Rule-based algorithms are required in safety-critical applications for them to work properly in a wide range of situations. Hence, every effort is made to find failure scenarios during the development phase. However, as the software becomes complicated, finding failure cases becomes difficult. Especially in multi-agent domains, such as autonomous driving environments, it is much harder to find useful failure scenarios that help us improve the algorithm. We propose a method for efficiently finding failure scenarios; this method trains the adversarial agents using multi-agent reinforcement learning such that the tested rule-based agent fails. We demonstrate the effectiveness of our proposed method using a simple environment and autonomous driving simulator.

scholarly journals Coupling SUMO with a Motion Planning Framework for Automated Vehicles

10.29007/1p2d ◽  
2019 ◽  
Author(s):  
Moritz Klischat ◽  
Octav Dragoi ◽  
Mostafa Eissa ◽  
Matthias Althoff
Keyword(s):  
Motion Planning ◽  
Open Source ◽  
Dynamic Behavior ◽  
Real World ◽  
Software Framework ◽  
Automated Vehicles ◽  
Lane Changing ◽  
Realistic Simulation ◽  
Lateral Dynamics ◽  
Planning Framework

Testing motion planning algorithms for automated vehicles in realistic simulation environments accelerates their development compared to performing real-world test drives only. In this work, we combine the open-source microscopic traffic simulator SUMO with our software framework CommonRoad to test motion planning of automated vehicles. Since SUMO is not originally designed for simulating automated vehicles, we present an inter- face for exchanging the trajectories of vehicles controlled by a motion planner and the trajectories of other traffic participants between SUMO and CommonRoad. Furthermore, we ensure realistic dynamic behavior of other traffic participants by extending the lane changing model in SUMO to implement more realistic lateral dynamics. We demonstrate our SUMO interface with a highway scenario.

Multi-agent Deep Reinforcement Learning for Autonomous Driving

2018 ◽  
Vol 24 (12) ◽  
pp. 670-674
Author(s):  
Hongsuk Yi ◽  
Eunsoo Park ◽  
Seungil Kim
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Driving ◽  
Multi Agent

scholarly journals CarFree: Hassle-Free Object Detection Dataset Generation Using Carla Autonomous Driving Simulator

2021 ◽  
Vol 12 (1) ◽  
pp. 281
Author(s):  
Jaesung Jang ◽  
Hyeongyu Lee ◽  
Jong-Chan Kim
Keyword(s):  
Object Detection ◽  
Real World ◽  
Driving Simulator ◽  
Data Extraction ◽  
Ground Truth ◽  
Autonomous Driving ◽  
Relevant Information ◽  
Detection Accuracy ◽  
Daunting Task ◽  
Bounding Boxes

For safe autonomous driving, deep neural network (DNN)-based perception systems play essential roles, where a vast amount of driving images should be manually collected and labeled with ground truth (GT) for training and validation purposes. After observing the manual GT generation’s high cost and unavoidable human errors, this study presents an open-source automatic GT generation tool, CarFree, based on the Carla autonomous driving simulator. By that, we aim to democratize the daunting task of (in particular) object detection dataset generation, which was only possible by big companies or institutes due to its high cost. CarFree comprises (i) a data extraction client that automatically collects relevant information from the Carla simulator’s server and (ii) a post-processing software that produces precise 2D bounding boxes of vehicles and pedestrians on the gathered driving images. Our evaluation results show that CarFree can generate a considerable amount of realistic driving images along with their GTs in a reasonable time. Moreover, using the synthesized training images with artificially made unusual weather and lighting conditions, which are difficult to obtain in real-world driving scenarios, CarFree significantly improves the object detection accuracy in the real world, particularly in the case of harsh environments. With CarFree, we expect its users to generate a variety of object detection datasets in hassle-free ways.

scholarly journals Application of Multi-Agent Deep Reinforcement Learning to Optimize Real-World Traffic Signal Controls

2021 ◽  
Author(s):  
Maxim Friesen ◽  
Tian Tan ◽  
Jürgen Jasperneite ◽  
Jie Wang
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Traffic Congestion ◽  
Traffic Signal ◽  
Traffic Model ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Logic ◽  
Rule Based ◽  
Multi Agent

Increasing traffic congestion leads to significant costs associated by additional travel delays, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today's traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that could operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, most of these proposed approaches were validated using artificial traffic grids. This paper therefore presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, currently employed in the real world at the studied intersections, are integrated in the traffic model with LISA+ and serve as a performance baseline. Our performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach in LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.

scholarly journals Multi-agent Reinforcement Learning for Autonomous Vehicles in Wireless Sensor Networks

2021 ◽  
Author(s):  
Haitham Afifi
Keyword(s):  
Wireless Sensor Networks ◽  
Reinforcement Learning ◽  
Sensor Networks ◽  
Real World ◽  
Autonomous Vehicles ◽  
Wireless Sensor ◽  
Acoustic Sensor Networks ◽  
Markov Decision ◽  
Multi Agent

<div>We develop a Deep Reinforcement Learning (DeepRL) based multi-agent algorithm to efficiently control</div><div>autonomous vehicles in the context of Wireless Sensor Networks (WSNs). In contrast to other applications, WSNs</div><div>have two metrics for performance evaluation. First, quality of information (QoI) which is used to measure the</div><div>quality of sensed data. Second, quality of service (QoS) which is used to measure the network’s performance. As</div><div>a use case, we consider wireless acoustic sensor networks; a group of speakers move inside a room and there</div><div>are microphones installed on vehicles for streaming the audio data. We formulate an appropriate Markov Decision</div><div>Process (MDP) and present, besides a centralized solution, a multi-agent Deep Q-learning solution to control the vehicles. We compare the proposed solutions to a naive heuristic and two different real-world implementations: microphones being hold or preinstalled. We show using simulations that the performance of autonomous vehicles in terms of QoI and QoS is better than the real-world implementation and the proposed heuristic. Additionally, we provide theoretical analysis of the performance with respect to WSNs dynamics, such as speed, rooms dimensions and speaker’s talking time.</div>

Sign in / Sign up

Export Citation Format

Share Document