A Reinforcement Learning Neural Network for Robotic Manipulator Control

2018 ◽  
Vol 30 (7) ◽  
pp. 1983-2004 ◽  
Author(s):  
Yazhou Hu ◽  
Bailu Si
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Performance Index ◽  
Robotic Manipulator ◽  
The State ◽  
Unknown Parameters ◽  
Proposed Model ◽  
Action Network ◽  
Action Policy ◽  
The Stability

We propose a neural network model for reinforcement learning to control a robotic manipulator with unknown parameters and dead zones. The model is composed of three networks. The state of the robotic manipulator is predicted by the state network of the model, the action policy is learned by the action network, and the performance index of the action policy is estimated by a critic network. The three networks work together to optimize the performance index based on the reinforcement learning control scheme. The convergence of the learning methods is analyzed. Application of the proposed model on a simulated two-link robotic manipulator demonstrates the effectiveness and the stability of the model.

scholarly journals A Stable Distributed Neural Controller for Physically Coupled Networked Discrete-Time System via Online Reinforcement Learning

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Jian Sun ◽  
Jie Li
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Large Scale ◽  
Controller Design ◽  
Learning Algorithm ◽  
Tracking Error ◽  
Discrete Time System ◽  
Learning Mechanisms ◽  
Action Network ◽  
The Stability

The large scale, time varying, and diversification of physically coupled networked infrastructures such as power grid and transportation system lead to the complexity of their controller design, implementation, and expansion. For tackling these challenges, we suggest an online distributed reinforcement learning control algorithm with the one-layer neural network for each subsystem or called agents to adapt the variation of the networked infrastructures. Each controller includes a critic network and action network for approximating strategy utility function and desired control law, respectively. For avoiding a large number of trials and improving the stability, the training of action network introduces supervised learning mechanisms into reduction of long-term cost. The stability of the control system with learning algorithm is analyzed; the upper bound of the tracking error and neural network weights are also estimated. The effectiveness of our proposed controller is illustrated in the simulation; the results indicate the stability under communication delay and disturbances as well.

Location- and Person-Independent Activity Recognition with WiFi, Deep Neural Networks, and Reinforcement Learning

2021 ◽  
Vol 2 (1) ◽  
pp. 1-25
Author(s):  
Yongsen Ma ◽  
Sheheryar Arshad ◽  
Swetha Muniraju ◽  
Eric Torkildson ◽  
Enrico Rantala ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reinforcement Learning ◽  
Activity Recognition ◽  
Deep Neural Networks ◽  
State Machine ◽  
Recognition Algorithm ◽  
The State ◽  
Neural Architecture ◽  
Learning Agent

In recent years, Channel State Information (CSI) measured by WiFi is widely used for human activity recognition. In this article, we propose a deep learning design for location- and person-independent activity recognition with WiFi. The proposed design consists of three Deep Neural Networks (DNNs): a 2D Convolutional Neural Network (CNN) as the recognition algorithm, a 1D CNN as the state machine, and a reinforcement learning agent for neural architecture search. The recognition algorithm learns location- and person-independent features from different perspectives of CSI data. The state machine learns temporal dependency information from history classification results. The reinforcement learning agent optimizes the neural architecture of the recognition algorithm using a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM). The proposed design is evaluated in a lab environment with different WiFi device locations, antenna orientations, sitting/standing/walking locations/orientations, and multiple persons. The proposed design has 97% average accuracy when testing devices and persons are not seen during training. The proposed design is also evaluated by two public datasets with accuracy of 80% and 83%. The proposed design needs very little human efforts for ground truth labeling, feature engineering, signal processing, and tuning of learning parameters and hyperparameters.

Force Control of Robot Manipulator by Neural Network Model - Experiment and Evaluation of One-Degree-of-Freedom Manipulator -

1990 ◽  
Vol 2 (4) ◽  
pp. 273-281 ◽  
Author(s):  
Masatoshi Tokita ◽  
◽  
Toyokazu Mitsuoka ◽  
Toshio Fukuda ◽  
Takashi Kurihara ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Force Control ◽  
Pid Controller ◽  
Robot Manipulator ◽  
Force Sensor ◽  
Robotic Manipulator ◽  
The Neural Network ◽  
The Stability

In this paper, a force control of a robotic manipulator based on a neural network model is proposed with consideration of the dynamics of both the force sensor and objects. This proposed system consists of the standard PID controller, the gains of which are augmented and adjusted depending on objects through a process of learning. The authors proposed a similar method previously for the force control of the robotic manipulator with consideration of dynamics of objects, but without consideration of dynamics of the force sensor, showing only simulation results. This paper shows the similar structure of the controller via the neural network model applicable to the cases with consideration of both effects and demonstrates that the proposed method shows the better performance than the conventional PID type of controller, yielding to the wider range of applications, consequently. Therefore, this method can be applied to the force/compliance control problems. The effects of the number of neurons and hidden layers of the neural network model are also discussed through the simulation and experimental results as well as the stability of the control system.

Neural Network Sliding Mode Control of 5DOFs Robotic Manipulator

2014 ◽  
Vol 898 ◽  
pp. 514-520 ◽  
Author(s):  
Chang Kai Xu ◽  
Ming Li ◽  
Jian Yin
Keyword(s):  
Neural Network ◽  
Sliding Mode Control ◽  
Sliding Mode ◽  
Learning Algorithm ◽  
Rbf Neural Network ◽  
Dynamic Performance ◽  
Robotic Manipulator ◽  
Terminal Sliding Mode ◽  
Mode Control ◽  
The Stability

In this paper, a neural network sliding mode controller for a kind of 5DOFs robotic manipulator is proposed. A radial basis function (RBF) neural network is used as an estimator to approximate uncertainties of the system. The learning algorithm of the neural network improves the performance of the system. A globle terminal sliding mode control (GTSMC) is designed to guarantee the stability and improve the dynamic performance of the robotic manipulator. Simulation results show that the proposed NNSMC strategy is effective to ensure the robustness and dynamic performance of the 5DOFs robotic manipulator.

scholarly journals Neural Network-Based Adaptive Control of Robotic Manipulator: Application to a Three Links Cylindrical Robot

2017 ◽  
Vol 13 (1) ◽  
pp. 114-122
Author(s):  
Abdul-Basset AL-Hussein
Keyword(s):  
Neural Network ◽  
Sliding Mode ◽  
Robot Manipulator ◽  
Tracking Error ◽  
Adaptive Controller ◽  
Robotic Manipulator ◽  
Lyapunov Theorem ◽  
External Disturbances ◽  
The Stability ◽  
Learning Law

A composite PD and sliding mode neural network (NN)-based adaptive controller, for robotic manipulator trajectory tracking, is presented in this paper. The designed neural networks are exploited to approximate the robotics dynamics nonlinearities, and compensate its effect and this will enhance the performance of the filtered error based PD and sliding mode controller. Lyapunov theorem has been used to prove the stability of the system and the tracking error boundedness. The augmented Lyapunov function is used to derive the NN weights learning law. To reduce the effect of breaching the NN learning law excitation condition due to external disturbances and measurement noise; a modified learning law is suggested based on e-modification algorithm. The controller effectiveness is demonstrated through computer simulation of cylindrical robot manipulator.

scholarly journals Deep-Reinforcement Learning-Based Co-Evolution in a Predator–Prey System

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 773 ◽  
Author(s):  
Xueting Wang ◽  
Jun Cheng ◽  
Lei Wang
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Behavior Pattern ◽  
Learning Ability ◽  
Predator Prey ◽  
Complex Processes ◽  
Proposed Model ◽  
The Stability ◽  
Reasonable Behavior ◽  
Evolutionary Networks

Understanding or estimating the co-evolution processes is critical in ecology, but very challenging. Traditional methods are difficult to deal with the complex processes of evolution and to predict their consequences on nature. In this paper, we use the deep-reinforcement learning algorithms to endow the organism with learning ability, and simulate their evolution process by using the Monte Carlo simulation algorithm in a large-scale ecosystem. The combination of the two algorithms allows organisms to use experiences to determine their behavior through interaction with that environment, and to pass on experience to their offspring. Our research showed that the predators’ reinforcement learning ability contributed to the stability of the ecosystem and helped predators obtain a more reasonable behavior pattern of coexistence with its prey. The reinforcement learning effect of prey on its own population was not as good as that of predators and increased the risk of extinction of predators. The inconsistent learning periods and speed of prey and predators aggravated that risk. The co-evolution of the two species had resulted in fewer numbers of their populations due to their potentially antagonistic evolutionary networks. If the learnable predators and prey invade an ecosystem at the same time, prey had an advantage. Thus, the proposed model illustrates the influence of learning mechanism on a predator–prey ecosystem and demonstrates the feasibility of predicting the behavior evolution in a predator–prey ecosystem using AI approaches.

An Enhanced Recurrent Convolutional Neural Network for Predicting the Status Stage of Patients with Chronic obstructive Pulmonary Diseases: Method Design (Preprint)

2021 ◽  
Author(s):  
Wei Li ◽  
Tianyong Hao ◽  
Zhanjie Mai ◽  
Pengjiu Yu ◽  
Chunli Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Real World ◽  
The State ◽  
Disease Stage ◽  
Chronic Obstructive ◽  
Clinical Dataset ◽  
Copd Patients ◽  
Proposed Model ◽  
The Status

BACKGROUND Chronic Obstructive Pulmonary Disease (COPD) is the third leading cause of death in China and has caused serious affect to health and life quality. However, the status stage of a patient is difficult to be accurately assessed because of dynamic changes in the condition and complex risk factors. A rapid and accurate methods to predict disease stage of COPD patients is of great significance. OBJECTIVE This study aims to explore an enhanced recurrent convolutional neural networks model for predicting correct staging of patients with COPD in China for assistant disease prevention and treatment. METHODS Data was collected from The First Affiliate Hospital of Guangzhou Medical University, which had standardized disease registration and follow-up management for 5108 patients with COPD. Our enhanced recurrent convolutional neural network consists of a bidirectional LSTM layer, a convolutional layer, a max-pooling layer, and an output layer. RESULTS The model proposed was evaluated on the real-world clinical dataset of 5108 COPD patients to predict the state stage of the disease. The performance of the proposed model achieved 93.2% in terms of accuracy, outperforming a list of baseline models. CONCLUSIONS This paper proposes an enhanced recurrent convolutional neural network model which is experimented on a real-world clinical dataset containing around 5,000 patients with COPD. The proposed model achieves the best performance on all evaluation metrics indicating its feasibility in predicting the state stage of diseases.

Intelligent scheduling using a neural network model in conjunction with reinforcement learning

2005 ◽  
Vol 219 (2) ◽  
pp. 229-235
Author(s):  
C J Fourie
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Reinforcement Learning ◽  
Learning From Experience ◽  
Scheduling System ◽  
Simulated Environment ◽  
The Neural Network ◽  
Learning Techniques ◽  
Proposed Model ◽  
Intelligent Scheduling

This paper describes the use of an artificial neural network in conjunction with reinforcement learning techniques to develop an intelligent scheduling system that is capable of learning from experience. In a simulated environment the model controls a mobile robot that transports material to machines. States of ‘happiness’ are defined for each machine, which are the inputs to the neural network. The output of the neural network is the decision on which machine to service next. After every decision, a critic evaluates the decision and a teacher ‘rewards’ the network to encourage good decisions and discourage bad decisions. From the results obtained, it is concluded that the proposed model is capable of learning from past experience and thereby improving the intelligence of the system.

scholarly journals Application of Multi-Dimension Input Convolutional Neural Network in Fault Diagnosis of Rolling Bearings

2019 ◽  
Vol 9 (13) ◽  
pp. 2690 ◽  
Author(s):  
Tao Zan ◽  
Hui Wang ◽  
Min Wang ◽  
Zhihao Liu ◽  
Xiangsheng Gao
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Convolutional Neural Network ◽  
Recognition Rate ◽  
Rolling Bearing ◽  
Diagnostic Model ◽  
Original Signal ◽  
Proposed Model ◽  
Convolution Model ◽  
The Stability

Aiming at the problem of poor robustness of the intelligent diagnostic model, a fault diagnosis model of rolling bearing based on a multi-dimension input convolutional neural network (MDI-CNN) is proposed. Compared with the traditional convolution neural network, the proposed model has multiple input layers. Therefore, it can fuse the original signal and processed signal—making full use of advantages of the convolutional neural networks to learn the original signal characteristics automatically, and also improving recognition accuracy and anti-jamming ability. The feasibility and validity of the proposed MDI-CNN are verified, and its advantages are proved by comparison with the other related models. Moreover, the robustness of the model is tested by adding the noise to the test set. Finally, the stability of the model is verified by two experiments. The experimental results show that the proposed model improves the recognition rate, robustness and convergence performance of the traditional convolution model and has good generalization ability.

Sign in / Sign up

Export Citation Format

Share Document