Vibration Suppression for Large-Scale Flexible Structures Using Deep Reinforcement Learning Based on Cable-Driven Parallel Robots

2020 ◽  
Author(s):  
Haining Sun ◽  
Xiaoqiang Tang ◽  
Jinhao Wei
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Vibration Suppression ◽  
External Disturbance ◽  
Flexible Structures ◽  
Parallel Robot ◽  
Parallel Robots ◽  
Gradient Algorithm ◽  
Normal Operation ◽  
Control Scheme

Abstract Specific satellites with ultra-long wings play a crucial role in many fields. However, external disturbance and self-rotation could result in undesired vibrations of flexible wings, which affects the normal operation of the satellites. In severe cases, the satellites will be damaged. Therefore, it is imperative to conduct vibration suppression for these flexible structures. Utilizing deep reinforcement learning (DRL), an active control scheme is presented in this paper to rapidly suppress the vibration of flexible structures with quite small controllable force based on a cable-driven parallel robot (CDPR). To verify the controller’s effectiveness, three groups of simulation with different initial disturbance are implemented. Besides, to enhance the contrast, a passive pre-tightening scheme is also tested. First, the dynamic model of the CDPR that is comprised of four cables and a flexible structure is established using the finite element method. Then, the dynamic behavior of the model under the controllable cable force is analyzed by Newmark-ß method. Furthermore, the agent of DRL is trained by the deep deterministic policy gradient algorithm (DDPG). Finally, the control scheme is conducted on Simulink environment to evaluate its performance, and the results are satisfactory, which validates the controller’s ability to suppress vibrations.

Vibration suppression for large-scale flexible structures based on cable-driven parallel robots

2020 ◽  
pp. 107754632096194
Author(s):  
Haining Sun ◽  
Xiaoqiang Tang ◽  
Senhao Hou ◽  
Xiaoyu Wang
Keyword(s):  
Large Scale ◽  
Vibration Suppression ◽  
Control Method ◽  
External Disturbance ◽  
Flexible Structures ◽  
Parallel Robot ◽  
Parallel Robots ◽  
Lyapunov Theory ◽  
Normal Operation ◽  
Control Methods

Specific satellites with ultralong wings play a crucial role in many fields. However, external disturbance and self-rotation could result in undesired vibrations of the flexible wings, which affect the normal operation of the satellites. In severe cases, the satellites would be damaged. Therefore, it is imperative to conduct vibration suppression for these flexible structures. Utilizing fuzzy-proportional integral derivative control and deep reinforcement learning (DRL), two active control methods are proposed in this article to rapidly suppress the vibration of flexible structures with quite small controllable force based on a cable-driven parallel robot. Inspired by the output law of DRL, a new control method named Tang and Sun control is innovatively presented based on the Lyapunov theory. To verify the effectiveness of these three control methods, three groups of simulations with different initial disturbances are implemented for each method. Besides, to enhance the contrast, a passive pretightening scheme is also tested. First, the dynamic model of the cable-driven parallel robot which comprises four cables and a flexible structure is established using the finite element method. Then, the dynamic behavior of the model under the controllable cable force is analyzed by the Newmark-ß method. Finally, these control methods are implemented by numerical simulations to evaluate their performance, and the results are satisfactory, which validates the controllers’ ability to suppress vibrations.

scholarly journals Research on a LADRC Strategy for Trajectory Tracking Control of Delta High-Speed Parallel Robots

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Cheng Liu ◽  
Yanming Cheng ◽  
Dejun Liu ◽  
Guohua Cao ◽  
Ilkyoo Lee
Keyword(s):  
Trajectory Tracking ◽  
High Speed ◽  
External Disturbance ◽  
Parallel Robot ◽  
Parallel Robots ◽  
System Stability ◽  
External Disturbances ◽  
Trajectory Tracking Control ◽  
Disturbance Rejection Control ◽  
Control Scheme

In order to better track the planned trajectory of Delta high-speed parallel robot, this paper proposes a dynamics control strategy for Delta high-speed parallel robots based on the linear active disturbance rejection control (LADRC) strategy which realizes decoupling control through observing and compensating the coupling and internal and external disturbances between the three joints. Firstly, the structure and dynamics model of the Delta high-speed parallel robot are analyzed, respectively. Secondly, the control scheme of the Delta high-speed parallel robot dynamic LADRC strategy is constructed, and then, the system stability is analyzed. Taking a representative 8-shaped space helical variance trajectory as a given input of the system and a triangular wave as an external disturbance as given disturbance input of the system, simulations are carried out to demonstrate the effectiveness of the proposed LADRC strategy; results indicate that the system with the LADRC strategy has a good quick and precise real-time trajectory tracking and strong robustness.

scholarly journals Robust Quadrotor Control through Reinforcement Learning with Disturbance Compensation

2021 ◽  
Vol 11 (7) ◽  
pp. 3257
Author(s):  
Chen-Huan Pi ◽  
Wei-Yuan Ye ◽  
Stone Cheng
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Control Strategy ◽  
External Disturbance ◽  
Control Agent ◽  
Network Control ◽  
Outdoor Environment ◽  
Disturbance Compensation ◽  
Tracking Accuracy ◽  
Control Scheme

In this paper, a novel control strategy is presented for reinforcement learning with disturbance compensation to solve the problem of quadrotor positioning under external disturbance. The proposed control scheme applies a trained neural-network-based reinforcement learning agent to control the quadrotor, and its output is directly mapped to four actuators in an end-to-end manner. The proposed control scheme constructs a disturbance observer to estimate the external forces exerted on the three axes of the quadrotor, such as wind gusts in an outdoor environment. By introducing an interference compensator into the neural network control agent, the tracking accuracy and robustness were significantly increased in indoor and outdoor experiments. The experimental results indicate that the proposed control strategy is highly robust to external disturbances. In the experiments, compensation improved control accuracy and reduced positioning error by 75%. To the best of our knowledge, this study is the first to achieve quadrotor positioning control through low-level reinforcement learning by using a global positioning system in an outdoor environment.

scholarly journals Reinforcement-Learning-Based Asynchronous Formation Control Scheme for Multiple Unmanned Surface Vehicles

2021 ◽  
Vol 11 (2) ◽  
pp. 546
Author(s):  
Jiajia Xie ◽  
Rui Zhou ◽  
Yuan Liu ◽  
Jun Luo ◽  
Shaorong Xie ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Formation Control ◽  
Rapid Development ◽  
Gradient Algorithm ◽  
Robot System ◽  
Physical Relationship ◽  
Unmanned Surface Vehicles ◽  
Main Challenge ◽  
Control Scheme ◽  
Multi Robot

The high performance and efficiency of multiple unmanned surface vehicles (multi-USV) promote the further civilian and military applications of coordinated USV. As the basis of multiple USVs’ cooperative work, considerable attention has been spent on developing the decentralized formation control of the USV swarm. Formation control of multiple USV belongs to the geometric problems of a multi-robot system. The main challenge is the way to generate and maintain the formation of a multi-robot system. The rapid development of reinforcement learning provides us with a new solution to deal with these problems. In this paper, we introduce a decentralized structure of the multi-USV system and employ reinforcement learning to deal with the formation control of a multi-USV system in a leader–follower topology. Therefore, we propose an asynchronous decentralized formation control scheme based on reinforcement learning for multiple USVs. First, a simplified USV model is established. Simultaneously, the formation shape model is built to provide formation parameters and to describe the physical relationship between USVs. Second, the advantage deep deterministic policy gradient algorithm (ADDPG) is proposed. Third, formation generation policies and formation maintenance policies based on the ADDPG are proposed to form and maintain the given geometry structure of the team of USVs during movement. Moreover, three new reward functions are designed and utilized to promote policy learning. Finally, various experiments are conducted to validate the performance of the proposed formation control scheme. Simulation results and contrast experiments demonstrate the efficiency and stability of the formation control scheme.

Tendon Actuated Continuous Structures in Planar Parallel Robots: A Kinematic Analysis

2020 ◽  
Vol 13 (1) ◽  
Author(s):  
Sven Lilge ◽  
Kathrin Nuelle ◽  
Georg Boettcher ◽  
Svenja Spindeldreier ◽  
Jessica Burgner-Kahrs
Keyword(s):  
Recent Work ◽  
Constant Curvature ◽  
Kinematic Analysis ◽  
Flexible Structures ◽  
Parallel Robot ◽  
Parallel Robots ◽  
Continuum Robots ◽  
Advantages And Disadvantages ◽  
Reference Design ◽  
Exact Quantification

Abstract The use of continuous and flexible structures instead of rigid links and discrete joints is a growing field of robotics research. Recent work focuses on the inclusion of continuous segments in parallel robots to benefit from their structural advantages, such as a high dexterity and compliance. While some applications and designs of these novel parallel continuum robots have been presented, the field remains largely unexplored. Furthermore, an exact quantification of the kinematic advantages and disadvantages when using continuous structures in parallel robots is yet to be performed. In this paper, planar parallel robot designs using tendon actuated continuum robots instead of rigid links and discrete joints are proposed. Using the well-known 3-RRR manipulator as a reference design, two parallel continuum robots are derived. Inverse and differential kinematics of these designs are modeled using constant curvature assumptions, which can be adapted for other actuation mechanisms than tendons. Their kinematic performances are compared to the conventional parallel robot counterpart. On the basis of this comparison, the advantages and disadvantages of using continuous structures in parallel robots are quantified and analyzed. Results show that parallel continuum robots can be kinematic equivalent and exhibit similar kinematic performances in comparison to conventional parallel robots depending on the chosen design.

Optimal Trajectory Tracking Control With a 5R Parallel Robot

2011 ◽  
Author(s):  
Gianmarc Coppola ◽  
Dan Zhang
Keyword(s):  
Trajectory Tracking ◽  
Feedforward Neural Networks ◽  
Parameter Tuning ◽  
Parallel Robot ◽  
Parallel Robots ◽  
Automatic Tuning ◽  
Trajectory Tracking Control ◽  
Model Reference Control ◽  
Control Scheme ◽  
Optimal Trajectory Tracking Control

This work examines the control characteristics of a 5R parallel robotic manipulator subjected to two control studies. Firstly, fundamental aspects of dynamics are presented. Then a brief review of Particle Swarm Optimization (PSO) and feedforward Neural Networks (NN) is undertaken. Subsequently, to tackle the challenging problem of controller parameter tuning for parallel robots in trajectory tracking scenarios, a multi objective optimization problem is formulated for automatic tuning using PSO. This offline method is comparatively evaluated to the Nelder-Mead (NM) sequential simplex optimization scheme. Several results are attained illustrating the strengths and weaknesses of this method for parallel robot control. Then, an adaptive NN model reference control scheme using PSO is proposed. This scheme is proposed as one possible way to take advantage of the strong properties of the PSO online. The scheme is tested and several observations are outlined.

An adaptive PID-like controller for vibration suppression of piezo-actuated flexible beams

2017 ◽  
Vol 24 (12) ◽  
pp. 2656-2670 ◽  
Author(s):  
Teerawat Sangpet ◽  
Suwat Kuntanapreeda ◽  
Rüdiger Schmidt
Keyword(s):  
Vibration Suppression ◽  
Flexible Structures ◽  
Flexible Beam ◽  
Flexible Beams ◽  
Present Scheme ◽  
Infinite Dimensional ◽  
Infinite Dimensional Systems ◽  
Finite Dimensional ◽  
Control Scheme ◽  
The Stability

Flexible structures have been increasingly utilized in many applications because of their light-weight and low production cost. However, being flexible leads to vibration problems. Vibration suppression of flexible structures is a challenging control problem because the structures are actually infinite-dimensional systems. In this paper, an adaptive control scheme is proposed for the vibration suppression of a piezo-actuated flexible beam. The controller makes use of the configuration of the prominent proportional-integral-derivative controller and is derived using an infinite-dimensional Lyapunov method. In contrast to existing schemes, the present scheme does not require any approximated finite-dimensional model of the beam. Thus, the stability of the closed loop system is guaranteed for all vibration modes. Experimental results have illustrated the feasibility of the proposed control scheme.

scholarly journals Revising the Observation Satellite Scheduling Problem Based on Deep Reinforcement Learning

2021 ◽  
Vol 13 (12) ◽  
pp. 2377
Author(s):  
Yixin Huang ◽  
Zhongcheng Mu ◽  
Shufan Wu ◽  
Benjie Cui ◽  
Yuxiao Duan
Keyword(s):  
Reinforcement Learning ◽  
Task Scheduling ◽  
Large Scale ◽  
Gradient Algorithm ◽  
Experimental Simulation ◽  
Scheduling Problem ◽  
Policy Gradient ◽  
Scheduling Method ◽  
Satellite Scheduling ◽  
Metaheuristic Optimization Algorithms

Earth observation satellite task scheduling research plays a key role in space-based remote sensing services. An effective task scheduling strategy can maximize the utilization of satellite resources and obtain larger objective observation profits. In this paper, inspired by the success of deep reinforcement learning in optimization domains, the deep deterministic policy gradient algorithm is adopted to solve a time-continuous satellite task scheduling problem. Moreover, an improved graph-based minimum clique partition algorithm is proposed for preprocessing in the task clustering phase by considering the maximum task priority and the minimum observation slewing angle under constraint conditions. Experimental simulation results demonstrate that the deep reinforcement learning-based task scheduling method is feasible and performs much better than traditional metaheuristic optimization algorithms, especially in large-scale problems.

scholarly journals Decentralized Offloading Strategies Based on Reinforcement Learning for Multi-Access Edge Computing

Information ◽  
2021 ◽  
Vol 12 (9) ◽  
pp. 343
Author(s):  
Chunyang Hu ◽  
Jingchen Li ◽  
Haobin Shi ◽  
Bin Ning ◽  
Qiong Gu
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Learning Model ◽  
Learning Technologies ◽  
Edge Computing ◽  
Gradient Algorithm ◽  
Computing Systems ◽  
Decentralized Execution ◽  
Multi Access ◽  
Reinforcement Learning Model

Using reinforcement learning technologies to learn offloading strategies for multi-access edge computing systems has been developed by researchers. However, large-scale systems are unsuitable for reinforcement learning, due to their huge state spaces and offloading behaviors. For this reason, this work introduces the centralized training and decentralized execution mechanism, designing a decentralized reinforcement learning model for multi-access edge computing systems. Considering a cloud server and several edge servers, we separate the training and execution in the reinforcement learning model. The execution happens in edge devices of the system, and edge servers need no communication. Conversely, the training process occurs at the cloud device, which causes a lower transmission latency. The developed method uses a deep deterministic policy gradient algorithm to optimize offloading strategies. The simulated experiment shows that our method can learn the offloading strategy for each edge device efficiently.

Sign in / Sign up

Export Citation Format

Share Document