Incremental Learning Framework for Autonomous Robots based on Q-learning and the Adaptive Kernel Linear Model

Reinforcement learning is one of the most promising machine learning techniques to get intelligent behaviors for embodied agents in simulations. The output of the classic Temporal Difference family of Reinforcement Learning algorithms adopts the form of a value function expressed as a numeric table or a function approximator. The learned behavior is then derived using a greedy policy with respect to this value function. Nevertheless, sometimes the learned policy does not meet expectations, and the task of authoring is difficult and unsafe because the modification of one value or parameter in the learned value function has unpredictable consequences in the space of the policies it represents. This invalidates direct manipulation of the learned value function as a method to modify the derived behaviors. In this paper, we propose the use of Inverse Reinforcement Learning to incorporate real behavior traces in the learning process to shape the learned behaviors, thus increasing their trustworthiness (in terms of conformance to reality). To do so, we adapt the Inverse Reinforcement Learning framework to the navigation problem domain. Specifically, we use Soft Q-learning, an algorithm based on the maximum causal entropy principle, with MARL-Ped (a Reinforcement Learning-based pedestrian simulator) to include information from trajectories of real pedestrians in the process of learning how to navigate inside a virtual 3D space that represents the real environment. A comparison with the behaviors learned using a Reinforcement Learning classic algorithm (Sarsa(λ)) shows that the Inverse Reinforcement Learning behaviors adjust significantly better to the real trajectories.

Download Full-text

Frequency-Temporal Disagreement Adaptation for Robotic Terrain Classification via Vibration in a Dynamic Environment

Sensors ◽

10.3390/s20226550 ◽

2020 ◽

Vol 20 (22) ◽

pp. 6550 ◽

Cited By ~ 1

Author(s):

Chen Cheng ◽

Ji Chang ◽

Wenjun Lv ◽

Yuping Wu ◽

Kun Li ◽

...

Keyword(s):

Incremental Learning ◽

Classification Accuracy ◽

Concept Drift ◽

Dynamic Environment ◽

Temporal Correlation ◽

Classification Methods ◽

Terrain Classification ◽

Localization Accuracy ◽

Learning Framework ◽

Control Scheme

The accurate terrain classification in real time is of great importance to an autonomous robot working in field, because the robot could avoid non-geometric hazards, adjust control scheme, or improve localization accuracy, with the aid of terrain classification. In this paper, we investigate the vibration-based terrain classification (VTC) in a dynamic environment, and propose a novel learning framework, named DyVTC, which tackles online-collected unlabeled data with concept drift. In the DyVTC framework, the exterior disagreement (ex-disagreement) and interior disagreement (in-disagreement) are proposed novely based on the feature diversity and intrinsic temporal correlation, respectively. Such a disagreement mechanism is utilized to design a pseudo-labeling algorithm, which shows its compelling advantages in extracting key samples and labeling; and consequently, the classification accuracy could be retrieved by incremental learning in a changing environment. Since two sets of features are extracted from frequency and time domain to generate disagreements, we also name the proposed method feature-temporal disagreement adaptation (FTDA). The real-world experiment shows that the proposed DyVTC could reach an accuracy of 89.5%, but the traditional time- and frequency-domain terrain classification methods could only reach 48.8% and 71.5%, respectively, in a dynamic environment.

Download Full-text

A Semi-Supervised Incremental Learning Framework for Sports Video View Classification

2006 12th International Multi-Media Modelling Conference ◽

10.1109/mmmc.2006.1651302 ◽

2006 ◽

Author(s):

Jun Wu ◽

Bo Zhang ◽

Xian-Sheng Hua ◽

Jianwei Zhang

Keyword(s):

Incremental Learning ◽

Sports Video ◽

Learning Framework

Download Full-text

Grounding of Human Environments and Activities for Autonomous Robots

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/193 ◽

2017 ◽

Author(s):

Muhannad Alomari ◽

Paul Duckworth ◽

Nils Bore ◽

Majd Hawasly ◽

David C. Hogg ◽

...

Keyword(s):

Natural Language ◽

Real World ◽

Human Activities ◽

Autonomous Robots ◽

Proof Of Concept ◽

Learning Framework ◽

Art Object ◽

Simple Sentences ◽

Robotic Applications ◽

Continual Learning

With the recent proliferation of human-oriented robotic applications in domestic and industrial scenarios, it is vital for robots to continually learn about their environments and about the humans they share their environments with. In this paper, we present a novel, online, incremental framework for unsupervised symbol grounding in real-world, human environments for autonomous robots. We demonstrate the flexibility of the framework by learning about colours, people names, usable objects and simple human activities, integrating state-of-the-art object segmentation, pose estimation, activity analysis along with a number of sensory input encodings into a continual learning framework. Natural language is grounded to the learned concepts, enabling the robot to communicate in a human-understandable way. We show, using a challenging real-world dataset of human activities as perceived by a mobile robot, that our framework is able to extract useful concepts, ground natural language descriptions to them, and, as a proof-of-concept, generate simple sentences from templates to describe people and the activities they are engaged in.

Download Full-text

Multi-Robot Cooperative Transportation of Objects Using Modified Q-Learning

Volume 8: Dynamic Systems and Control, Parts A and B ◽

10.1115/imece2010-40055 ◽

2010 ◽

Author(s):

Pallege Gamini Dilupa Siriwardana ◽

Clarence de Silva

Keyword(s):

Autonomous Robots ◽

Dynamic Environment ◽

Learning Approach ◽

Robot System ◽

Q Learning ◽

Time Performance ◽

Object Pose Estimation ◽

Object Transportation ◽

Blob Tracking ◽

Multi Robot

In cooperative multi-robot object transportation, several autonomous robots navigate cooperatively in either a static or a dynamic environment to transport an object to a goal location and orientation. The environment may consist of both fixed and removable obstacles and it will be subject to uncertainty and unforeseen changes within the environment. More than one robot may be required for handling heavy and large objects. This paper presents a modified Q-learning approach for object transportation utilizing cooperative and autonomous multiple mobile robots. A modified version of Q-learning is presented, which employs for effective robot coordination. A solution to the action selection conflicts of the robots is presented, which helps to improve the real time performance and robustness of the system. As required in the task, the paper presents an algorithm for object pose estimation, by utilizing the laser range finder and color blob tracking. The developed techniques are implemented in a multi-robot system in laboratory. Experimental results are presented to demonstrate the effectiveness of the developed multi-robot system and its underlying methodologies.

Download Full-text