scholarly journals Decentralized Coordinated Motion Control of Two Hydraulic Actuators Handling a Common Object

2007 ◽  
Vol 129 (5) ◽  
pp. 729-741 ◽  
Author(s):  
Mark Karpenko ◽  
Nariman Sepehri ◽  
John Anderson
Keyword(s):  
Reinforcement Learning ◽  
Position Control ◽  
Control Strategies ◽  
Interaction Force ◽  
Learning System ◽  
Multiagent Learning ◽  
Credit Assignment ◽  
Interaction Forces ◽  
Actuator Dynamics ◽  
Hydraulic Actuators

In this paper, reinforcement learning is applied to coordinate, in a decentralized fashion, the motions of a pair of hydraulic actuators whose task is to firmly hold and move an object along a specified trajectory under conventional position control. The learning goal is to reduce the interaction forces acting on the object that arise due to inevitable positioning errors resulting from the imperfect closed-loop actuator dynamics. Each actuator is therefore outfitted with a reinforcement learning neural network that modifies a centrally planned formation constrained position trajectory in response to the locally measured interaction force. It is shown that the actuators, which form a multiagent learning system, can learn decentralized control strategies that reduce the object interaction forces and thus greatly improve their coordination on the manipulation task. However, the problem of credit assignment, a common difficulty in multiagent learning systems, prevents the actuators from learning control strategies where each actuator contributes equally to reducing the interaction force. This problem is resolved in this paper via the periodic communication of limited local state information between the reinforcement learning actuators. Using both simulations and experiments, this paper examines some of the issues pertaining to learning in dynamic multiagent environments and establishes reinforcement learning as a potential technique for coordinating several nonlinear hydraulic manipulators performing a common task.

Learning and control

2018 ◽  
Author(s):  
Ivan Herreros
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Brain Function ◽  
Control Strategies ◽  
Learning Problems ◽  
Animal Learning ◽  
Feed Forward Control ◽  
Machine Learning Applications ◽  
And Control

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

Energy consumption optimization for AC drives position control

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Branislav Ftorek ◽  
Milan Saga ◽  
Pavol Orsansky ◽  
Jan Vittek ◽  
Peter Butko
Keyword(s):  
Control System ◽  
Energy Saving ◽  
Energy Savings ◽  
Position Control ◽  
Control Strategies ◽  
Content Type ◽  
Ac Drives ◽  
Energy Demands ◽  
Wide Range ◽  
Energy Consumption Optimization

Purpose The main purpose of this paper is to evaluate the two energy saving position control strategies for AC drives valid for a wide range of boundary conditions including an analysis of their energy expenses. Design/methodology/approach For energy demands analysis, the optimal energy control based on mechanical and electrical losses minimization is compared with the near-optimal one based on symmetrical trapezoidal speed profile. Both control strategies respect prescribed maneuver time and define acceleration profile for preplanned rest-to-rest maneuver. Findings Presented simulations confirm lower total energy expenditures of energy optimal control if compared with near-optimal one, but the differences are only small due to the fact that two energy saving strategies are compared. Research limitations/implications Developed overall control system consisting of energy saving profile generator, pre-compensator and position control system respecting principles of field-oriented control is capable to track precomputed state variables precisely. Practical implications Energy demands of both control strategies are verified and compared to simulations and preliminary experiments. The possibilities of energy savings were confirmed for both control strategies. Originality/value Experimental verification of designed control structure is sufficiently promising and confirmed assumed energy savings.

scholarly journals Novel Design and Position Control Strategy of a Soft Robot Arm

Robotics ◽  
2018 ◽  
Vol 7 (4) ◽  
pp. 72 ◽  
Author(s):  
Alaa Al-Ibadi ◽  
Samia Nefti-Meziani ◽  
Steve Davis ◽  
Theo Theodoridis
Keyword(s):  
Position Control ◽  
Control Strategies ◽  
Tensile Force ◽  
Compressive Force ◽  
Soft Robot ◽  
Robot Arm ◽  
Bending Actuator ◽  
Muscle Actuators ◽  
Novel Design ◽  
Bending Behaviour

This article presents a novel design of a continuum arm, which has the ability to extend and bend efficiently. Numerous designs and experiments have been done to different dimensions on both types of McKibben pneumatic muscle actuators (PMA) in order to study their performances. The contraction and extension behaviour have been illustrated with single contractor actuators and single extensor actuators, respectively. The tensile force for the contractor actuator and the compressive force for the extensor PMA are thoroughly explained and compared. Furthermore, the bending behaviour has been explained for a single extensor PMA, multi extensor actuators and multi contractor actuators. A two-section continuum arm has been implemented from both types of actuators to achieve multiple operations. Then, a novel construction is proposed to achieve efficient bending behaviour of a single contraction PMA. This novel design of a bending-actuator has been used to modify the presented continuum arm. Two different position control strategies are presented, arising from the results of the modified soft robot arm experiment. A cascaded position control is applied to control the position of the end effector of the soft arm at no load by efficiently controlling the pressure of all the actuators in the continuum arm. A new algorithm is then proposed by distributing the x, y and z-axis to the actuators and applying an effective closed-loop position control to the proposed arm at different load conditions.

scholarly journals Mental state space visualization for interactive modeling of personalized BCI control strategies

2019 ◽  
Author(s):  
Ilya Kuzovkin ◽  
Konstantin Tretyakov ◽  
Andero Uusberg ◽  
Raul Vicente
Keyword(s):  
State Space ◽  
Mental State ◽  
Control Strategies ◽  
Muscular Activity ◽  
Mental States ◽  
Learning System ◽  
Self Organizing Map ◽  
Preliminary Validation ◽  
Som Algorithm ◽  
Mental Actions

AbstractObjectiveNumerous studies in the area of BCI are focused on the search for a better experimental paradigm – a set of mental actions that a user can evoke consistently and a machine can discriminate reliably. Examples of such mental activities are motor imagery, mental computations, etc. We propose a technique that instead allows the user to try different mental actions in the search for the ones that will work best.ApproachThe system is based on a modification of the self-organizing map (SOM) algorithm and enables interactive communication between the user and the learning system through a visualization of user’s mental state space. During the interaction with the system the user converges on the paradigm that is most efficient and intuitive for that particular user.Main resultsResults of the two experiments, one allowing muscular activity, another permitting mental activity only, demonstrate soundness of the proposed method and offer preliminary validation of the performance improvement over the traditional closed-loop feedback approach.SignificanceThe proposed method allows a user to visually explore their mental state space in real time, opening new opportunities for scientific inquiry. The application of this method to the area of brain-computer interfaces enables more efficient search for the mental states that will allow a user to reliably control a BCI system.

scholarly journals Self-Organized Fuzzy Reinforcement Learning System

2003 ◽  
Vol 39 (7) ◽  
pp. 699-701
Author(s):  
Kosuke UMESAKO ◽  
Masanao OBAYASHI ◽  
Kunikazu KOBAYASHI
Keyword(s):  
Reinforcement Learning ◽  
Learning System ◽  
Self Organized

Accurate Position Control of a Pneumatic Actuator

1990 ◽  
Vol 112 (4) ◽  
pp. 734-739 ◽  
Author(s):  
Jiing-Yih Lai ◽  
Chia-Hsiang Menq ◽  
Rajendra Singh
Keyword(s):  
Control Strategy ◽  
Pulse Width Modulation ◽  
Position Control ◽  
Integral Control ◽  
Pneumatic Actuators ◽  
Actuator Dynamics ◽  
Position Accuracy ◽  
Modulation Mode ◽  
Nonlinear Mechanical ◽  
Accurate Position

We propose a new control strategy for on-off valve controlled pneumatic actuators and robots with focus on the position accuracy. A mathematical model incorporating pneumatic process nonlinearities and nonlinear mechanical friction has been developed to characterize the actuator dynamics; this model with a few simplifications is then used to design the controller. In our control scheme, one valve is held open and the other is operated under the pulse width modulation mode to simulate the proportional control. An inner loop utilizing proportional-plus-integral control is formed to control the actuator pressure, and an outer loop with displacement and velocity feedbacks is used to control the load displacement. Also, a two staged feedforward force is implemented to reduce the steady state error due to the nonlinear mechanical friction. Experimental results on a single-degree-of-freedom pneumatic robot indicate that the proposed control system is better than the conventional on-off control strategy as it is effective in achieving the desired position accuracy without using any mechanical stops in the actuator.

scholarly journals Patient-Centered Robot-Aided Passive Neurorehabilitation Exercise Based on Safety-Motion Decision-Making Mechanism

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Lizheng Pan ◽  
Aiguo Song ◽  
Suolin Duan ◽  
Zhuqing Yu
Keyword(s):  
Decision Making ◽  
Control Strategy ◽  
Position Control ◽  
Impedance Control ◽  
Control Strategies ◽  
Movement Speed ◽  
Stroke Patients ◽  
Patient Centered ◽  
Motion Speed ◽  
Clinical Experiments

Safety is one of the crucial issues for robot-aided neurorehabilitation exercise. When it comes to the passive rehabilitation training for stroke patients, the existing control strategies are usually just based on position control to carry out the training, and the patient is out of the controller. However, to some extent, the patient should be taken as a “cooperator” of the training activity, and the movement speed and range of the training movement should be dynamically regulated according to the internal or external state of the subject, just as what the therapist does in clinical therapy. This research presents a novel motion control strategy for patient-centered robot-aided passive neurorehabilitation exercise from the point of the safety. The safety-motion decision-making mechanism is developed to online observe and assess the physical state of training impaired-limb and motion performances and regulate the training parameters (motion speed and training rage), ensuring the safety of the supplied rehabilitation exercise. Meanwhile, position-based impedance control is employed to realize the trajectory tracking motion with interactive compliance. Functional experiments and clinical experiments are investigated with a healthy adult and four recruited stroke patients, respectively. The two types of experimental results demonstrate that the suggested control strategy not only serves with safety-motion training but also presents rehabilitation efficacy.

Robust Walking Control of a Lower Limb Rehabilitation Exoskeleton Coupled with a Musculoskeletal Model via Deep Reinforcement Learning

2021 ◽  
Author(s):  
Shuzhen Luo ◽  
Ghaith Androwis ◽  
Sergei Adamovich ◽  
Erick Nunez ◽  
Hao Su ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Neuromuscular Disorders ◽  
Control Policy ◽  
Policy Network ◽  
Control Parameters ◽  
Robust Controller ◽  
Interaction Forces ◽  
Learning Framework ◽  
Limb Rehabilitation ◽  
Lower Limb Rehabilitation

Abstract Background: Few studies have systematically investigated robust controllers for lower limb rehabilitation exoskeletons (LLREs) that can safely and effectively assist users with a variety of neuromuscular disorders to walk with full autonomy. One of the key challenges for developing such a robust controller is to handle different degrees of uncertain human-exoskeleton interaction forces from the patients. Consequently, conventional walking controllers either are patient-condition specific or involve tuning of many control parameters, which could behave unreliably and even fail to maintain balance. Methods: We present a novel and robust controller for a LLRE based on a decoupled deep reinforcement learning framework with three independent networks, which aims to provide reliable walking assistance against various and uncertain human-exoskeleton interaction forces. The exoskeleton controller is driven by a neural network control policy that acts on a stream of the LLRE’s proprioceptive signals, including joint kinematic states, and subsequently predicts real-time position control targets for the actuated joints. To handle uncertain human-interaction forces, the control policy is trained intentionally with an integrated human musculoskeletal model and realistic human-exoskeleton interaction forces. Two other neural networks are connected with the control policy network to predict the interaction forces and muscle coordination. To further increase the robustness of the control policy, we employ domain randomization during training that includes not only randomization of exoskeleton dynamics properties but, more importantly, randomization of human muscle strength to simulate the variability of the patient’s disability. Through this decoupled deep reinforcement learning framework, the trained controller of LLREs is able to provide reliable walking assistance to the human with different degrees of neuromuscular disorders. Results and Conclusion: A universal, RL-based walking controller is trained and virtually tested on a LLRE system to verify its effectiveness and robustness in assisting users with different disabilities such as passive muscles (quadriplegic), muscle weakness, or hemiplegic conditions. An ablation study demonstrates strong robustness of the control policy under large exoskeleton dynamic property ranges and various human-exoskeleton interaction forces. The decoupled network structure allows us to isolate the LLRE control policy network for testing and sim-to-real transfer since it uses only proprioception information of the LLRE (joint sensory state) as the input. Furthermore, the controller is shown to be able to handle different patient conditions without the need for patient-specific control parameters tuning.

Sign in / Sign up

Export Citation Format

Share Document