Motion generation for walking exoskeleton robot using multiple dynamic movement primitives sequences combined with reinforcement learning

In order to assist patients with lower limb disabilities in normal walking, a new trajectory learning scheme of limb exoskeleton robot based on dynamic movement primitives (DMP) combined with reinforcement learning (RL) was proposed. The developed exoskeleton robot has six degrees of freedom (DOFs). The hip and knee of each artificial leg can provide two electric-powered DOFs for flexion/extension. And two passive-installed DOFs of the ankle were used to achieve the motion of inversion/eversion and plantarflexion/dorsiflexion. The five-point segmented gait planning strategy is proposed to generate gait trajectories. The gait Zero Moment Point stability margin is used as a parameter to construct a stability criteria to ensure the stability of human-exoskeleton system. Based on the segmented gait trajectory planning formation strategy, the multiple-DMP sequences were proposed to model the generation trajectories. Meanwhile, in order to eliminate the effect of uncertainties in joint space, the RL was adopted to learn the trajectories. The experiment demonstrated that the proposed scheme can effectively remove interferences and uncertainties.

A Novel Balance Control Strategy Based on Enhanced Stability Pyramid Index and Dynamic Movement Primitives for a Lower Limb Human-Exoskeleton System

The lower limb exoskeleton is playing an increasing role in enabling individuals with spinal cord injury (SCI) to stand upright, walk, turn, and so on. Hence, it is essential to maintain the balance of the human-exoskeleton system during movements. However, the balance of the human-exoskeleton system is challenging to maintain. There are no effective balance control strategies because most of them can only be used in a specific movement like walking or standing. Hence, the primary aim of the current study is to propose a balance control strategy to improve the balance of the human-exoskeleton system in dynamic movements. This study proposes a new safety index named Enhanced Stability Pyramid Index (ESPI), and a new balance control strategy is based on the ESPI and the Dynamic Movement Primitives (DMPs). To incorporate dynamic information of the system, the ESPI employs eXtrapolated Center of Mass (XCoM) instead of the center of mass (CoM). Meanwhile, Time-to-Contact (TTC), the urgency of safety, is used as an automatic weight assignment factor of ESPI instead of the traditional manual one. Then, the balance control strategy utilizing DMPs to generate the gait trajectory according to the scalar and vector values of the ESPI is proposed. Finally, the walking simulation in Gazebo and the experiments of the human-exoskeleton system verify the effectiveness of the index and balance control strategy.

Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance

Dynamic movement primitives (DMPs) are a robust framework for movement generation from demonstrations. This framework can be extended by adding a perturbing term to achieve obstacle avoidance without sacrificing stability. The additional term is usually constructed based on potential functions. Although different potentials are adopted to improve the performance of obstacle avoidance, the profiles of potentials are rarely incorporated into reinforcement learning (RL) framework. In this contribution, we present a RL based method to learn not only the profiles of potentials but also the shape parameters of a motion. The algorithm employed is PI2 (Policy Improvement with Path Integrals), a model-free, sampling-based learning method. By using the PI2, the profiles of potentials and the parameters of the DMPs are learned simultaneously; therefore, we can optimize obstacle avoidance while completing specified tasks. We validate the presented method in simulations and with a redundant robot arm in experiments.

Movement generalization of variable initial task state based on Euclidean transformation dynamical movement primitives

Robots need the ability to tackle problems of movement generalization in variable task state and complex environment. Dynamical movement primitives can effectively endow robots with humanoid characteristics. However, when the initial state of tasks changes, the generalized trajectories by dynamical movement primitives cannot retain shape features of demonstration, resulting in the loss of imitation quality. In this article, a modified dynamical movement primitives based on Euclidean transformation is proposed to solve this problem. It transforms the initial task state to a virtual situation similar to the demonstration and then utilizes the dynamical movement primitive method to realize movement generalization. Finally, it reverses the movement back to the real situation. Besides, the information of obstacles is added to Euclidean transformation based dynamical movement primitives framework to endow robots with the ability of obstacle avoidance. The normalized root-mean-square error is proposed as the criterion to evaluate the imitation similarity. The feasibility of this method is verified through writing letters, wiping whiteboard in two-dimensional task, and stirring mixture in three-dimensional task. The results show that the similarity of movement imitation in the proposed method is higher than dynamical movement primitives when the initial state changes. Meanwhile, Euclidean transformation based dynamical movement primitives can still greatly retain shape feature of demonstration while avoiding obstacles in an unstructured environment.

Enhancing learning capabilities of movement primitives under distributed probabilistic framework for flexible assembly tasks

This paper presents a novel probabilistic distributed framework based on movement primitives for flexible robot assembly. Since the modern advanced industrial cell usually deals with various scenarios that are not fixed via-point trajectories but highly reconfigurable tasks, the industrial robots used in these applications must be capable of adapting and learning new in-demand skills without programming experts. Therefore, we propose a probabilistic framework that could accommodate various learning abilities trained with different movement-primitive datasets, separately. Derived from the Bayesian Committee Machine, this framework could infer new adapting trajectories with weighted contributions of each training dataset. To verify the feasibility of our proposed imitation learning framework, the simulation comparison with the state-of-the-art movement learning framework task-parametrised GMM is conducted. Several key aspects, such as generalisation capability, learning accuracy and computation expense, are discussed and compared. Moreover, two real-world experiments, i.e. riveting picking and nutplate picking, are further tested with the YuMi collaborative robot to verify the application feasibility in industrial assembly manufacturing.