scholarly journals Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges

Robotics ◽  
2019 ◽  
Vol 8 (1) ◽  
pp. 4 ◽  
Author(s):  
Sarthak Bhagat ◽  
Hritwick Banerjee ◽  
Zion Ho Tse ◽  
Hongliang Ren
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Degrees Of Freedom ◽  
Review Article ◽  
Imitation Learning ◽  
Robotic Systems ◽  
Soft Robotics ◽  
Flexible Robots ◽  
Increasing Trend ◽  
Embodied Intelligence

The increasing trend of studying the innate softness of robotic structures and amalgamating it with the benefits of the extensive developments in the field of embodied intelligence has led to the sprouting of a relatively new yet rewarding sphere of technology in intelligent soft robotics. The fusion of deep reinforcement algorithms with soft bio-inspired structures positively directs to a fruitful prospect of designing completely self-sufficient agents that are capable of learning from observations collected from their environment. For soft robotic structures possessing countless degrees of freedom, it is at times not convenient to formulate mathematical models necessary for training a deep reinforcement learning (DRL) agent. Deploying current imitation learning algorithms on soft robotic systems has provided competent results. This review article posits an overview of various such algorithms along with instances of being applied to real-world scenarios, yielding frontier results. Brief descriptions highlight the various pristine branches of DRL research in soft robotics.

scholarly journals Deep Reinforcement Learning for Soft Robotic Applications: Brief Overview with Impending Challenges

2018 ◽  
Author(s):  
Sarthak Bhagat ◽  
Hritwick Banerjee ◽  
Hongliang Ren
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
State Of The Art ◽  
Imitation Learning ◽  
Robotic Systems ◽  
Future Research ◽  
Learning Techniques ◽  
Increasing Trend ◽  
Embodied Intelligence ◽  
Robotic Applications

The increasing trend of studying the innate softness of robotic structures and amalgamating it with the benefits of the extensive developments in the field of embodied intelligence has led to sprouting of a relatively new yet extremely rewarding sphere of technology. The fusion of current deep reinforcement algorithms with physical advantages of a soft bio-inspired structure certainly directs us to a fruitful prospect of designing completely self-sufficient agents that are capable of learning from observations collected from their environment to achieve a task they have been assigned. For soft robotics structure possessing countless degrees of freedom, it is often not easy (something not even possible) to formulate mathematical constraints necessary for training a deep reinforcement learning (DRL) agent for the task in hand, hence, we resolve to imitation learning techniques due to ease of manually performing such tasks like manipulation that could be comfortably mimicked by our agent. Deploying current imitation learning algorithms on soft robotic systems have been observed to provide satisfactory results but there are still challenges in doing so. This review article thus posits an overview of various such algorithms along with instances of them being applied to real world scenarios and yielding state-of-the-art results followed by brief descriptions on various pristine branches of DRL research that may be centers of future research in this field of interest.

scholarly journals Deep Reinforcement Learning for Soft Robotic Applications: Brief Overview with Impending Challenges

2018 ◽  
Author(s):  
Sarthak Bhagat ◽  
Hritwick Banerjee ◽  
Hongliang Ren
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
State Of The Art ◽  
Imitation Learning ◽  
Robotic Systems ◽  
Future Research ◽  
Learning Techniques ◽  
Increasing Trend ◽  
Embodied Intelligence ◽  
Robotic Applications

The increasing trend of studying the innate softness of robotic structures and amalgamating it with the benefits of the extensive developments in the field of embodied intelligence has led to sprouting of a relatively new yet extremely rewarding sphere of technology. The fusion of current deep reinforcement algorithms with physical advantages of a soft bio-inspired structure certainly directs us to a fruitful prospect of designing completely self-sufficient agents that are capable of learning from observations collected from their environment to achieve a task they have been assigned. For soft robotics structure possessing countless degrees of freedom, it is often not easy (something not even possible) to formulate mathematical constraints necessary for training a deep reinforcement learning (DRL) agent for the task in hand, hence, we resolve to imitation learning techniques due to ease of manually performing such tasks like manipulation that could be comfortably mimicked by our agent. Deploying current imitation learning algorithms on soft robotic systems have been observed to provide satisfactory results but there are still challenges in doing so. This review article thus posits an overview of various such algorithms along with instances of them being applied to real world scenarios and yielding state-of-the-art results followed by brief descriptions on various pristine branches of DRL research that may be centers of future research in this field of interest.

scholarly journals Goal-driven active learning

2021 ◽  
Vol 35 (2) ◽  
Author(s):  
Nicolas Bougie ◽  
Ryutaro Ichise
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Learning Process ◽  
Real World ◽  
Imitation Learning ◽  
Learning Approaches ◽  
Wide Range ◽  
Fixed Set ◽  
Complex Decision Making ◽  
Complex Decision

AbstractDeep reinforcement learning methods have achieved significant successes in complex decision-making problems. In fact, they traditionally rely on well-designed extrinsic rewards, which limits their applicability to many real-world tasks where rewards are naturally sparse. While cloning behaviors provided by an expert is a promising approach to the exploration problem, learning from a fixed set of demonstrations may be impracticable due to lack of state coverage or distribution mismatch—when the learner’s goal deviates from the demonstrated behaviors. Besides, we are interested in learning how to reach a wide range of goals from the same set of demonstrations. In this work we propose a novel goal-conditioned method that leverages very small sets of goal-driven demonstrations to massively accelerate the learning process. Crucially, we introduce the concept of active goal-driven demonstrations to query the demonstrator only in hard-to-learn and uncertain regions of the state space. We further present a strategy for prioritizing sampling of goals where the disagreement between the expert and the policy is maximized. We evaluate our method on a variety of benchmark environments from the Mujoco domain. Experimental results show that our method outperforms prior imitation learning approaches in most of the tasks in terms of exploration efficiency and average scores.

scholarly journals How to train your robot with deep reinforcement learning: lessons we have learned

2021 ◽  
pp. 027836492098785
Author(s):  
Julian Ibarz ◽  
Jie Tan ◽  
Chelsea Finn ◽  
Mrinal Kalakrishnan ◽  
Peter Pastor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Case Studies ◽  
Real World ◽  
Review Article ◽  
The Real ◽  
Complex Skills ◽  
Real World Learning ◽  
Level Sensor ◽  
Embodied Agent

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

Cutting the Cord: Progress in Untethered Soft Robotics and Actuators

MRS Advances ◽  
2019 ◽  
Vol 4 (51-52) ◽  
pp. 2787-2804 ◽  
Author(s):  
Meng Li ◽  
Nicholas A Ostrovsky-Snider ◽  
Metin Sitti ◽  
Fiorenzo G Omenetto
Keyword(s):  
Real World ◽  
Material Science ◽  
Degrees Of Freedom ◽  
Research Interest ◽  
Soft Robotics ◽  
Stimuli Responsive ◽  
New Materials ◽  
Real World Applications ◽  
Soft Actuators ◽  
Rigid Skeleton

AbstractIn recent decades, increasing research interest has shifted from traditional rigid skeleton robotics to flexible, shape-programmable, environmentally adaptive and stimuli-responsive “soft robotics”. Within this discipline, soft-robots capable of untethered and/or remote-controlled operation are of particular interest given their utility for actuation in complex situations with larger range of mobility and higher degrees of freedom. The use of new materials and the development of advanced fabrication techniques enable better performance and expand the utility of such soft actuators, moving them towards real-world applications. This review outlines some recent advances in untethered soft robotics and actuators to illustrate the promise of these applications at the interface of material science and device engineering.

scholarly journals Review of the State-of-the-art Sewer Monitoring and Maintenance Systems Pune Municipal Corporation - A Case Study

TEM Journal ◽  
2021 ◽  
pp. 1500-1508
Author(s):  
Ravindra R. Patil ◽  
Saniya M. Ansari ◽  
Rajnish Kaur Calay ◽  
Mohamad Y. Mustafa
Keyword(s):  
Computer Vision ◽  
Real Time ◽  
Developing Country ◽  
Real World ◽  
State Of The Art ◽  
Robotic Systems ◽  
Increasing Trend ◽  
Maintenance Systems ◽  
Sewer Inspection

There is an increasing trend of using automated and robotic systems for the tasks that are hazardous or inconvenient and dirty for humans. Sewers maintenance and cleaning is such a task where robots are already being used for inspection of underground pipes for blockages and damage. This paper reviews the existing robotic systems and various platforms and algorithms along with their capabilities and limitations being discussed. A typical mid-size city in a developing country, Pune, India is selected in order to understand the concerns and identify the requirements for developing robotic systems for the same. It is found that major concern of sewers are blockages but there is not enough information on both real-time detection and removal of it with robotic systems. On-board processing with computer vision algorithms has not been efficiently utilized in terms of performance and determinations for real-world implementations of sewer robotic systems. The review highlights the available methodologies that can be utilized in developing sewer inspection and cleaning robotic systems.

Recent Advances in Reinforcement Learning for Traffic Signal Control

2021 ◽  
Vol 22 (2) ◽  
pp. 12-18 ◽  
Author(s):  
Hua Wei ◽  
Guanjie Zheng ◽  
Vikash Gayah ◽  
Zhenhui Li
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Intelligent Transportation Systems ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Methods ◽  
Advantages And Disadvantages ◽  
Recent Advances

Traffic signal control is an important and challenging real-world problem that has recently received a large amount of interest from both transportation and computer science communities. In this survey, we focus on investigating the recent advances in using reinforcement learning (RL) techniques to solve the traffic signal control problem. We classify the known approaches based on the RL techniques they use and provide a review of existing models with analysis on their advantages and disadvantages. Moreover, we give an overview of the simulation environments and experimental settings that have been developed to evaluate the traffic signal control methods. Finally, we explore future directions in the area of RLbased traffic signal control methods. We hope this survey could provide insights to researchers dealing with real-world applications in intelligent transportation systems

Challenges of real-world reinforcement learning: definitions, benchmarks and analysis

2021 ◽  
Author(s):  
Gabriel Dulac-Arnold ◽  
Nir Levine ◽  
Daniel J. Mankowitz ◽  
Jerry Li ◽  
Cosmin Paduraru ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World

scholarly journals Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

2021 ◽  
Author(s):  
Amarildo Likmeta ◽  
Alberto Maria Metelli ◽  
Giorgia Ramponi ◽  
Andrea Tirinzoni ◽  
Matteo Giuliani ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Real Life ◽  
User Preferences ◽  
Inverse Reinforcement Learning ◽  
Water Release ◽  
Reward Function ◽  
Model Free ◽  
Conflicting Objectives ◽  
Multiple Experts

AbstractIn real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

Sign in / Sign up

Export Citation Format

Share Document