An Adaptive Imitation Learning Framework for Robotic Complex Contact-Rich Insertion Tasks

Complex contact-rich insertion is a ubiquitous robotic manipulation skill and usually involves nonlinear and low-clearance insertion trajectories as well as varying force requirements. A hybrid trajectory and force learning framework can be utilized to generate high-quality trajectories by imitation learning and find suitable force control policies efficiently by reinforcement learning. However, with the mentioned approach, many human demonstrations are necessary to learn several tasks even when those tasks require topologically similar trajectories. Therefore, to reduce human repetitive teaching efforts for new tasks, we present an adaptive imitation framework for robot manipulation. The main contribution of this work is the development of a framework that introduces dynamic movement primitives into a hybrid trajectory and force learning framework to learn a specific class of complex contact-rich insertion tasks based on the trajectory profile of a single task instance belonging to the task class. Through experimental evaluations, we validate that the proposed framework is sample efficient, safer, and generalizes better at learning complex contact-rich insertion tasks on both simulation environments and on real hardware.

Motion generation for walking exoskeleton robot using multiple dynamic movement primitives sequences combined with reinforcement learning

In order to assist patients with lower limb disabilities in normal walking, a new trajectory learning scheme of limb exoskeleton robot based on dynamic movement primitives (DMP) combined with reinforcement learning (RL) was proposed. The developed exoskeleton robot has six degrees of freedom (DOFs). The hip and knee of each artificial leg can provide two electric-powered DOFs for flexion/extension. And two passive-installed DOFs of the ankle were used to achieve the motion of inversion/eversion and plantarflexion/dorsiflexion. The five-point segmented gait planning strategy is proposed to generate gait trajectories. The gait Zero Moment Point stability margin is used as a parameter to construct a stability criteria to ensure the stability of human-exoskeleton system. Based on the segmented gait trajectory planning formation strategy, the multiple-DMP sequences were proposed to model the generation trajectories. Meanwhile, in order to eliminate the effect of uncertainties in joint space, the RL was adopted to learn the trajectories. The experiment demonstrated that the proposed scheme can effectively remove interferences and uncertainties.

A Novel Balance Control Strategy Based on Enhanced Stability Pyramid Index and Dynamic Movement Primitives for a Lower Limb Human-Exoskeleton System

The lower limb exoskeleton is playing an increasing role in enabling individuals with spinal cord injury (SCI) to stand upright, walk, turn, and so on. Hence, it is essential to maintain the balance of the human-exoskeleton system during movements. However, the balance of the human-exoskeleton system is challenging to maintain. There are no effective balance control strategies because most of them can only be used in a specific movement like walking or standing. Hence, the primary aim of the current study is to propose a balance control strategy to improve the balance of the human-exoskeleton system in dynamic movements. This study proposes a new safety index named Enhanced Stability Pyramid Index (ESPI), and a new balance control strategy is based on the ESPI and the Dynamic Movement Primitives (DMPs). To incorporate dynamic information of the system, the ESPI employs eXtrapolated Center of Mass (XCoM) instead of the center of mass (CoM). Meanwhile, Time-to-Contact (TTC), the urgency of safety, is used as an automatic weight assignment factor of ESPI instead of the traditional manual one. Then, the balance control strategy utilizing DMPs to generate the gait trajectory according to the scalar and vector values of the ESPI is proposed. Finally, the walking simulation in Gazebo and the experiments of the human-exoskeleton system verify the effectiveness of the index and balance control strategy.

Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance

Dynamic movement primitives (DMPs) are a robust framework for movement generation from demonstrations. This framework can be extended by adding a perturbing term to achieve obstacle avoidance without sacrificing stability. The additional term is usually constructed based on potential functions. Although different potentials are adopted to improve the performance of obstacle avoidance, the profiles of potentials are rarely incorporated into reinforcement learning (RL) framework. In this contribution, we present a RL based method to learn not only the profiles of potentials but also the shape parameters of a motion. The algorithm employed is PI2 (Policy Improvement with Path Integrals), a model-free, sampling-based learning method. By using the PI2, the profiles of potentials and the parameters of the DMPs are learned simultaneously; therefore, we can optimize obstacle avoidance while completing specified tasks. We validate the presented method in simulations and with a redundant robot arm in experiments.

An Extended DMPs framework for Decoupled Quaternions Learning and generalization

Dynamic Movement Primitives (DMPs) as a robust and efficient framework has been studied widely for robot learning from demonstration. Classical DMPs framework mainly focuses on the movement learning in Cartesian or joint space, and can't properly represent end-effector orientation. In this paper, we present an Extended DMPs framework (EDMPs) both in Cartesian space and Riemannian manifolds for Quaternion-based orientations learning and generalization. Gaussian Mixture Model and Gaussian Mixture Regression are adopted as the initialization phase of EDMPs to handle multi-demonstrations and obtain their mean and covariance. Additionally, some evaluation indicators including reachability and similarity are defined to characterize the learning and generalization abilities of EDMPs. Finally, the quaternion-based orientations are successfully transferred from human to the robot, and a real-world experiment is conducted to verify the effectiveness of the proposed method. The experimental results reveal that the presented approach can learn and generalize multi-space parameters under multi-demonstrations.