scholarly journals A Traffic-Aware Federated Imitation Learning Framework for Motion Control at Unsignalized Intersections with Internet of Vehicles

Electronics ◽  
2021 ◽  
Vol 10 (24) ◽  
pp. 3050
Author(s):  
Tianhao Wu ◽  
Mingzhi Jiang ◽  
Yinhui Han ◽  
Zheng Yuan ◽  
Xinhang Li ◽  
...  
Keyword(s):  
Motion Control ◽  
Imitation Learning ◽  
Selection Strategy ◽  
Communication Overhead ◽  
Model Interaction ◽  
Internet Of Vehicles ◽  
Traffic Lights ◽  
Learning Framework ◽  
Unsignalized Intersections ◽  
Control Optimization

The wealth of data and the enhanced computation capabilities of Internet of Vehicles (IoV) enable the optimized motion control of vehicles passing through an intersection without traffic lights. However, more intersections and demands for privacy protection pose new challenges to motion control optimization. Federated Learning (FL) can protect privacy via model interaction in IoV, but traditional FL methods hardly deal with the transportation issue. To address the aforementioned issue, this study proposes a Traffic-Aware Federated Imitation learning framework for Motion Control (TAFI-MC), consisting of Vehicle Interactors (VIs), Edge Trainers (ETs), and a Cloud Aggregator (CA). An Imitation Learning (IL) algorithm is integrated into TAFI-MC to improve motion control. Furthermore, a loss-aware experience selection strategy is explored to reduce communication overhead between ETs and VIs. The experimental results show that the proposed TAFI-MC outperforms imitated rules in the respect of collision avoidance and driving comfort, and the experience selection strategy can reduce communication overheads while ensuring convergence.

scholarly journals An Adaptive Imitation Learning Framework for Robotic Complex Contact-Rich Insertion Tasks

2022 ◽  
Vol 8 ◽  
Author(s):  
Yan Wang ◽  
Cristian C. Beltran-Hernandez ◽  
Weiwei Wan ◽  
Kensuke Harada
Keyword(s):  
Force Control ◽  
Robotic Manipulation ◽  
Imitation Learning ◽  
Specific Class ◽  
High Quality ◽  
Single Task ◽  
Learning Framework ◽  
Dynamic Movement ◽  
Real Hardware ◽  
Dynamic Movement Primitives

Complex contact-rich insertion is a ubiquitous robotic manipulation skill and usually involves nonlinear and low-clearance insertion trajectories as well as varying force requirements. A hybrid trajectory and force learning framework can be utilized to generate high-quality trajectories by imitation learning and find suitable force control policies efficiently by reinforcement learning. However, with the mentioned approach, many human demonstrations are necessary to learn several tasks even when those tasks require topologically similar trajectories. Therefore, to reduce human repetitive teaching efforts for new tasks, we present an adaptive imitation framework for robot manipulation. The main contribution of this work is the development of a framework that introduces dynamic movement primitives into a hybrid trajectory and force learning framework to learn a specific class of complex contact-rich insertion tasks based on the trajectory profile of a single task instance belonging to the task class. Through experimental evaluations, we validate that the proposed framework is sample efficient, safer, and generalizes better at learning complex contact-rich insertion tasks on both simulation environments and on real hardware.

scholarly journals Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

2020 ◽  
Author(s):  
Cong Fei ◽  
Bin Wang ◽  
Yuzheng Zhuang ◽  
Zongzhang Zhang ◽  
Jianye Hao ◽  
...  
Keyword(s):  
Real World ◽  
Autonomous Vehicles ◽  
Data Augmentation ◽  
State Of The Art ◽  
Robot Learning ◽  
Imitation Learning ◽  
Proper Understanding ◽  
Learning Framework ◽  
Strategy Game ◽  
Real Time Strategy Game

Generative adversarial imitation learning (GAIL) has shown promising results by taking advantage of generative adversarial nets, especially in the field of robot learning. However, the requirement of isolated single modal demonstrations limits the scalability of the approach to real world scenarios such as autonomous vehicles' demand for a proper understanding of human drivers' behavior. In this paper, we propose a novel multi-modal GAIL framework, named Triple-GAIL, that is able to learn skill selection and imitation jointly from both expert demonstrations and continuously generated experiences with data augmentation purpose by introducing an auxiliary selector. We provide theoretical guarantees on the convergence to optima for both of the generator and the selector respectively. Experiments on real driver trajectories and real-time strategy game datasets demonstrate that Triple-GAIL can better fit multi-modal behaviors close to the demonstrators and outperforms state-of-the-art methods.

Qualitative Motion Control Optimization of the Pad Dressing Process for Bonnet Tool

2019 ◽  
Vol 24 (3) ◽  
pp. 1141-1152
Author(s):  
Ri Pan ◽  
Yuhang Tang ◽  
Bo Zhong ◽  
Dongju Chen ◽  
Zhenzhong Wang ◽  
...  
Keyword(s):  
Motion Control ◽  
Control Optimization

scholarly journals Authentication Protocols for Internet of Things: A Comprehensive Survey

2017 ◽  
Vol 2017 ◽  
pp. 1-41 ◽  
Author(s):  
Mohamed Amine Ferrag ◽  
Leandros A. Maglaras ◽  
Helge Janicke ◽  
Jianmin Jiang ◽  
Lei Shu
Keyword(s):  
Internet Of Things ◽  
Future Research ◽  
Communication Overhead ◽  
Authentication Protocols ◽  
Internet Of Vehicles ◽  
Current Survey ◽  
Comprehensive Survey ◽  
Future Research Directions ◽  
Target Environment ◽  
Verification Techniques

In this paper, a comprehensive survey of authentication protocols for Internet of Things (IoT) is presented. Specifically more than forty authentication protocols developed for or applied in the context of the IoT are selected and examined in detail. These protocols are categorized based on the target environment: (1) Machine to Machine Communications (M2M), (2) Internet of Vehicles (IoV), (3) Internet of Energy (IoE), and (4) Internet of Sensors (IoS). Threat models, countermeasures, and formal security verification techniques used in authentication protocols for the IoT are presented. In addition a taxonomy and comparison of authentication protocols that are developed for the IoT in terms of network model, specific security goals, main processes, computation complexity, and communication overhead are provided. Based on the current survey, open issues are identified and future research directions are proposed.

scholarly journals Multiobjective model-free learning for robot pathfinding with environmental disturbances

2019 ◽  
Vol 16 (6) ◽  
pp. 172988141988570 ◽  
Author(s):  
Changyun Wei ◽  
Fusheng Ni
Keyword(s):  
Reinforcement Learning ◽  
Action Selection ◽  
Two Dimensions ◽  
Selection Strategy ◽  
Sequential Decision ◽  
Environmental Disturbances ◽  
Learning Framework ◽  
Model Free ◽  
Conflicting Objectives ◽  
Save Energy

This article addresses the robot pathfinding problem with environmental disturbances, where a solution to this problem must consider potential risks inherent in an uncertain and stochastic environment. For example, the movements of an underwater robot can be seriously disturbed by ocean currents, and thus any applied control actions to the robot cannot exactly lead to the desired locations. Reinforcement learning is a formal methodology that has been extensively studied in many sequential decision-making domains with uncertainty, but most reinforcement learning algorithms consider only a single objective encoded by a scalar reward. However, the robot pathfinding problem with environmental disturbances naturally promotes multiple conflicting objectives. Specifically, in this work, the robot has to minimise its moving distance so as to save energy, and, moreover, it has to keep away from unsafe regions as far as possible. To this end, we first propose a multiobjective model-free learning framework, and then proceed to investigate an appropriate action selection strategy by improving a baseline with respect to two dimensions. To demonstrate the effectiveness of the proposed learning framework and evaluate the performance of three action selection strategies, we also carry out an empirical study in a simulated environment.

scholarly journals Influences of Waiting Time on Driver Behaviors While Implementing In-Vehicle Traffic Light for Priority-Controlled Unsignalized Intersections

2017 ◽  
Vol 2017 ◽  
pp. 1-12 ◽  
Author(s):  
Bo Yang ◽  
Rencheng Zheng ◽  
Tsutomu Kaizuka ◽  
Kimihiko Nakano
Keyword(s):  
Waiting Time ◽  
Driving Simulator ◽  
Lateral Acceleration ◽  
Traffic Light ◽  
Road Vehicles ◽  
Maximum Acceleration ◽  
Vehicle Traffic ◽  
Traffic Lights ◽  
Unsignalized Intersections ◽  
Minor Road

In-vehicle traffic lights that assist drivers in crossing intersections are in development; however, the availability of the in-vehicle traffic light will be limited if the waiting time of a vehicle is not considered in actual traffic conditions, especially at priority-controlled unsignalized intersections that normally consist of one major and two minor roads. The present study therefore investigated the effects of the waiting time on driver behaviors to improve the in-vehicle traffic light for the priority-controlled unsignalized intersections. Gap acceptance theory that considers the waiting time was adopted in the implementation of the in-vehicle traffic light, to assist minor-road drivers in passing through the intersections by selecting appropriate major-road gaps. A driving simulator experiment involving 12 participants was performed for the minor and major roads, by applying the in-vehicle traffic light with and without the consideration of waiting time. Results demonstrate that the maximum acceleration strokes of minor-road vehicles were significantly reduced, indicating a lower possibility of aggressive driving when the in-vehicle traffic light was applied while considering the waiting time. Meanwhile, an improved steering stability was observed from the driver behaviors at the intersections, as the maximum lateral acceleration of minor-road vehicles significantly decreased when the waiting time was considered.

scholarly journals A New Corpus and Imitation Learning Framework for Context-Dependent Semantic Parsing

2014 ◽  
Vol 2 ◽  
pp. 547-560 ◽  
Author(s):  
Andreas Vlachos ◽  
Stephen Clark
Keyword(s):  
Information System ◽  
Natural Language ◽  
Learning Algorithm ◽  
Imitation Learning ◽  
Semantic Parsing ◽  
Learning Framework ◽  
Test Sets ◽  
Context Dependent ◽  
Meaning Representation ◽  
Tourist Information

Semantic parsing is the task of translating natural language utterances into a machine-interpretable meaning representation. Most approaches to this task have been evaluated on a small number of existing corpora which assume that all utterances must be interpreted according to a database and typically ignore context. In this paper we present a new, publicly available corpus for context-dependent semantic parsing. The MRL used for the annotation was designed to support a portable, interactive tourist information system. We develop a semantic parser for this corpus by adapting the imitation learning algorithm DAgger without requiring alignment information during training. DAgger improves upon independently trained classifiers by 9.0 and 4.8 points in F-score on the development and test sets respectively.

Sign in / Sign up

Export Citation Format

Share Document