Constrained attractor selection using deep reinforcement learning

This study describes an approach for attractor selection (or multistability control) in nonlinear dynamical systems with constrained actuation. Attractor selection is obtained using two different deep reinforcement learning methods: (1) the cross-entropy method and (2) the deep deterministic policy gradient method. The framework and algorithms for applying these control methods are presented. Experiments were performed on a Duffing oscillator, as it is a classic nonlinear dynamical system with multiple attractors. Both methods achieve attractor selection under various control constraints. Although these methods have nearly identical success rates, the deep deterministic policy gradient method has the advantages of a high learning rate, low performance variance, and a smooth control approach. This study demonstrates the ability of two reinforcement learning approaches to achieve constrained attractor selection.

Muscle Coordination Control for an Asymmetrically Antagonistic-Driven Musculoskeletal Robot Using Attractor Selection

Recently, numerous musculoskeletal robots have been developed to realize the flexibility and dexterity analogous to human beings and animals. However, because the arrangement of many actuators is complex, the design of the control system for the robot is difficult and challenging. We believe that control methods inspired by living things are important in the development of the control systems for musculoskeletal robots. In this study, we propose a muscle coordination control method using attractor selection, a biologically inspired search method, for an antagonistic-driven musculoskeletal robot in which various muscles (monoarticular muscles and a polyarticular muscle) are arranged asymmetrically. First, muscle coordination control models for the musculoskeletal robot are built using virtual antagonistic muscle structures with a virtually symmetric muscle arrangement. Next, the attractor selection is applied to the control model and subsequently applied to the previous control model without muscle coordination to compare the control model’s performance. Finally, position control experiments are conducted, and the effectiveness of the proposed muscle coordination control and the virtual antagonistic muscle structure is evaluated.