scholarly journals Deep Reinforcement Learning for Soft Robotic Applications: Brief Overview with Impending Challenges

2018 ◽  
Author(s):  
Sarthak Bhagat ◽  
Hritwick Banerjee ◽  
Hongliang Ren
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
State Of The Art ◽  
Imitation Learning ◽  
Robotic Systems ◽  
Future Research ◽  
Learning Techniques ◽  
Increasing Trend ◽  
Embodied Intelligence ◽  
Robotic Applications

The increasing trend of studying the innate softness of robotic structures and amalgamating it with the benefits of the extensive developments in the field of embodied intelligence has led to sprouting of a relatively new yet extremely rewarding sphere of technology. The fusion of current deep reinforcement algorithms with physical advantages of a soft bio-inspired structure certainly directs us to a fruitful prospect of designing completely self-sufficient agents that are capable of learning from observations collected from their environment to achieve a task they have been assigned. For soft robotics structure possessing countless degrees of freedom, it is often not easy (something not even possible) to formulate mathematical constraints necessary for training a deep reinforcement learning (DRL) agent for the task in hand, hence, we resolve to imitation learning techniques due to ease of manually performing such tasks like manipulation that could be comfortably mimicked by our agent. Deploying current imitation learning algorithms on soft robotic systems have been observed to provide satisfactory results but there are still challenges in doing so. This review article thus posits an overview of various such algorithms along with instances of them being applied to real world scenarios and yielding state-of-the-art results followed by brief descriptions on various pristine branches of DRL research that may be centers of future research in this field of interest.

scholarly journals Deep Reinforcement Learning for Soft Robotic Applications: Brief Overview with Impending Challenges

2018 ◽  
Author(s):  
Sarthak Bhagat ◽  
Hritwick Banerjee ◽  
Hongliang Ren
Keyword(s):  
Reinforcement Learning ◽  
Degrees Of Freedom ◽  
State Of The Art ◽  
Imitation Learning ◽  
Robotic Systems ◽  
Future Research ◽  
Learning Techniques ◽  
Increasing Trend ◽  
Embodied Intelligence ◽  
Robotic Applications

The increasing trend of studying the innate softness of robotic structures and amalgamating it with the benefits of the extensive developments in the field of embodied intelligence has led to sprouting of a relatively new yet extremely rewarding sphere of technology. The fusion of current deep reinforcement algorithms with physical advantages of a soft bio-inspired structure certainly directs us to a fruitful prospect of designing completely self-sufficient agents that are capable of learning from observations collected from their environment to achieve a task they have been assigned. For soft robotics structure possessing countless degrees of freedom, it is often not easy (something not even possible) to formulate mathematical constraints necessary for training a deep reinforcement learning (DRL) agent for the task in hand, hence, we resolve to imitation learning techniques due to ease of manually performing such tasks like manipulation that could be comfortably mimicked by our agent. Deploying current imitation learning algorithms on soft robotic systems have been observed to provide satisfactory results but there are still challenges in doing so. This review article thus posits an overview of various such algorithms along with instances of them being applied to real world scenarios and yielding state-of-the-art results followed by brief descriptions on various pristine branches of DRL research that may be centers of future research in this field of interest.

scholarly journals Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges

Robotics ◽  
2019 ◽  
Vol 8 (1) ◽  
pp. 4 ◽  
Author(s):  
Sarthak Bhagat ◽  
Hritwick Banerjee ◽  
Zion Ho Tse ◽  
Hongliang Ren
Keyword(s):  
Reinforcement Learning ◽  
Real World ◽  
Degrees Of Freedom ◽  
Review Article ◽  
Imitation Learning ◽  
Robotic Systems ◽  
Soft Robotics ◽  
Flexible Robots ◽  
Increasing Trend ◽  
Embodied Intelligence

The increasing trend of studying the innate softness of robotic structures and amalgamating it with the benefits of the extensive developments in the field of embodied intelligence has led to the sprouting of a relatively new yet rewarding sphere of technology in intelligent soft robotics. The fusion of deep reinforcement algorithms with soft bio-inspired structures positively directs to a fruitful prospect of designing completely self-sufficient agents that are capable of learning from observations collected from their environment. For soft robotic structures possessing countless degrees of freedom, it is at times not convenient to formulate mathematical models necessary for training a deep reinforcement learning (DRL) agent. Deploying current imitation learning algorithms on soft robotic systems has provided competent results. This review article posits an overview of various such algorithms along with instances of being applied to real-world scenarios, yielding frontier results. Brief descriptions highlight the various pristine branches of DRL research in soft robotics.

scholarly journals Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1278
Author(s):  
Jiang Hua ◽  
Liangcai Zeng ◽  
Gongfa Li ◽  
Zhaojie Ju
Keyword(s):  
Reinforcement Learning ◽  
Transfer Learning ◽  
Robot Control ◽  
Learning Transfer ◽  
State Of The Art ◽  
Imitation Learning ◽  
Future Research ◽  
Intelligent Robot ◽  
Dexterous Manipulation ◽  
Research Challenges

Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed.

scholarly journals Applying a Deep Q Network for OpenAIs Car Racing Game

2020 ◽  
Author(s):  
Ali Fakhry
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Transfer Learning ◽  
State Of The Art ◽  
Learning Techniques ◽  
Car Racing ◽  
Custom Made ◽  
Learning Technique ◽  
Reward Threshold

The applications of Deep Q-Networks are seen throughout the field of reinforcement learning, a large subsect of machine learning. Using a classic environment from OpenAI, CarRacing-v0, a 2D car racing environment, alongside a custom based modification of the environment, a DQN, Deep Q-Network, was created to solve both the classic and custom environments. The environments are tested using custom made CNN architectures and applying transfer learning from Resnet18. While DQNs were state of the art years ago, using it for CarRacing-v0 appears somewhat unappealing and not as effective as other reinforcement learning techniques. Overall, while the model did train and the agent learned various parts of the environment, attempting to reach the reward threshold for the environment with this reinforcement learning technique seems problematic and difficult as other techniques would be more useful.

State of the Art and Key Design Challenges of Telesurgical Robotics

2019 ◽  
pp. 1100-1111
Author(s):  
Sajid Nisar ◽  
Osman Hasan
Keyword(s):  
State Of The Art ◽  
Robotic Systems ◽  
Future Research ◽  
Research Directions ◽  
Remote Locations ◽  
Comprehensive Survey ◽  
Future Research Directions ◽  
Surgical System ◽  
Surgical Operations ◽  
Design Challenges

Telesurgical robotic systems allow surgeons to perform surgical operations from remote locations with enhanced comfort and dexterity. Introduction of robotic technology has revolutionized operation theaters but its multidisciplinary nature and high associated costs pose significant challenges. This chapter provides a comprehensive survey of the current progress in the field of surgical robotics with a detailed discussion on various state-of-the-art telesurgical robotic systems. The key design approaches and challenges are identified, and their solutions are recommended. A set of parameters that can be used to assess usefulness of a telesurgical robot are discussed. Finally, guidelines for selection of a suitable surgical system and the future research directions are described.

scholarly journals Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 6722-6729 ◽  
Author(s):  
Ziming Li ◽  
Julia Kiseleva ◽  
Maarten De Rijke
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
The State ◽  
Experimental Results ◽  
Imitation Learning ◽  
Local Optimum ◽  
Inverse Reinforcement Learning ◽  
High Quality ◽  
Overall Performance

The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.

A Compact 3 Degree of Freedom Spherical Joint

2011 ◽  
Vol 3 (3) ◽  
Author(s):  
Mark L. Guckert ◽  
Michael D. Naish
Keyword(s):  
Control Systems ◽  
Degrees Of Freedom ◽  
Closed Loop ◽  
Degree Of Freedom ◽  
Robotic Systems ◽  
Closed Loop Control ◽  
Spherical Joint ◽  
Experimental Assessment ◽  
Loop Control ◽  
Robotic Applications

Spherical joints have evolved into a critical component of many robotic systems, often used to provide dexterity at the wrist of a manipulator. In this work, a novel 3 degree of freedom spherical joint is proposed, actuated by tendons that run along the surface of the sphere. The joint is mechanically simple and avoids mechanical singularities. The kinematics and mechanics of the joint are modeled and used to develop both open- and closed-loop control systems. Simulated and experimental assessment of the joint performance demonstrates that it can be successfully controlled in 3 degrees of freedom. It is expected that the joint will be a useful option in the development of emerging robotic applications, particularly those requiring miniaturization.

scholarly journals BOLeRo: Behavior optimization and learning for robots

2020 ◽  
Vol 17 (3) ◽  
pp. 172988142091374
Author(s):  
Alexander Fabisch ◽  
Malte Langosz ◽  
Frank Kirchner
Keyword(s):  
Reinforcement Learning ◽  
Evolutionary Algorithms ◽  
Open Source ◽  
State Of The Art ◽  
Learning Algorithms ◽  
Robotic Systems ◽  
Policy Search ◽  
Behavior Learning ◽  
And Behavior

Reinforcement learning and behavior optimization are becoming more and more popular in the field of robotics because algorithms are mature enough to tackle real problems in this domain. Robust implementations of state-of-the-art algorithms are often not publicly available though, and experiments are hardly reproducible because open-source implementations are often not available or are still in a stage of research code. Consequently, often it is infeasible to deploy these algorithms on robotic systems. BOLeRo closes this gap for policy search and evolutionary algorithms by delivering open-source implementations of behavior learning algorithms for robots. It is easy to integrate in robotic middlewares and it can be used to compare methods and develop prototypes in simulation.

State of the Art and Key Design Challenges of Telesurgical Robotics

2018 ◽  
pp. 6872-6881 ◽  
Author(s):  
Sajid Nisar ◽  
Osman Hasan
Keyword(s):  
State Of The Art ◽  
Robotic Systems ◽  
Future Research ◽  
Research Directions ◽  
Remote Locations ◽  
Comprehensive Survey ◽  
Future Research Directions ◽  
Surgical System ◽  
Surgical Operations ◽  
Design Challenges

Telesurgical robotic systems allow surgeons to perform surgical operations from remote locations with enhanced comfort and dexterity. Introduction of robotic technology has revolutionized operation theaters but it's multidisciplinary nature and high associated costs pose significant challenges. This chapter provides a comprehensive survey of the current progress in the field of surgical robotics with a detailed discussion on various state-of-the-art telesurgical robotic systems. The key design approaches and challenges are identified and their solutions are recommended. A set of parameters that can be used to assess usefulness of a telesurgical robot are discussed. Finally, guidelines for selection of a suitable surgical system and the future research directions are described.

scholarly journals Soft Robotics: Biological Inspiration, State of the Art, and Future Research

2008 ◽  
Vol 5 (3) ◽  
pp. 99-117 ◽  
Author(s):  
Deepak Trivedi ◽  
Christopher D. Rahn ◽  
William M. Kier ◽  
Ian D. Walker
Keyword(s):  
Degrees Of Freedom ◽  
State Of The Art ◽  
Future Research ◽  
Robot Design ◽  
The Novel ◽  
Soft Robots ◽  
Biological Inspiration ◽  
End Effectors ◽  
And Control ◽  
Complex Movement

Traditional robots have rigid underlying structures that limit their ability to interact with their environment. For example, conventional robot manipulators have rigid links and can manipulate objects using only their specialised end effectors. These robots often encounter difficulties operating in unstructured and highly congested environments. A variety of animals and plants exhibit complex movement with soft structures devoid of rigid components. Muscular hydrostats (e.g. octopus arms and elephant trunks) are almost entirely composed of muscle and connective tissue and plant cells can change shape when pressurised by osmosis. Researchers have been inspired by biology to design and build soft robots. With a soft structure and redundant degrees of freedom, these robots can be used for delicate tasks in cluttered and/or unstructured environments. This paper discusses the novel capabilities of soft robots, describes examples from nature that provide biological inspiration, surveys the state of the art and outlines existing challenges in soft robot design, modelling, fabrication and control.

Sign in / Sign up

Export Citation Format

Share Document