The Eminence Grise

2020 ◽  
pp. 61-69
Author(s):  
Thomas Boraud
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Social Behaviour ◽  
Dynamic Properties ◽  
Leading Role ◽  
Cortical Level ◽  
The Neural Network ◽  
Main Input ◽  
Controlling Behaviour

This chapter demonstrates how the cortex communicates with the basal ganglia and the thalamus progressively during the evolution of vertebrates. The telencephalic loop is involved in social behaviour and cognitive processes. In mammals, the telencephalic loop, in which the cortex now replaces the pallium, takes a leading role in controlling behaviour. Thus, the telencephalon, a very minor input of the basal ganglia in anamniots, gradually becomes the main input as evolution progresses. Ultimately, the resulting neural network possesses the same dynamic properties as those described in Chapter 5. The neural network is also able to perform reinforcement learning through the subcortical loop, and also to automatize some skills at the cortical level.

scholarly journals Adaptive Control Structure with Neural Data Processing Applied for Electrical Drive with Elastic Shaft

Energies ◽  
2021 ◽  
Vol 14 (12) ◽  
pp. 3389
Author(s):  
Marcin Kamiński ◽  
Krzysztof Szabat
Keyword(s):  
Neural Network ◽  
Adaptive Control ◽  
Dynamic Properties ◽  
Experimental Studies ◽  
Drive System ◽  
Particle Swarm Optimizer ◽  
The Neural Network ◽  
Control Concept ◽  
Constant Coefficients ◽  
Selection Of

This paper presents issues related to the adaptive control of the drive system with an elastic clutch connecting the main motor and the load machine. Firstly, the problems and the main algorithms often implemented for the mentioned object are analyzed. Then, the control concept based on the RNN (recurrent neural network) for the drive system with the flexible coupling is thoroughly described. For this purpose, an adaptive model inspired by the Elman model is selected, which is related to internal feedback in the neural network. The indicated feature improves the processing of dynamic signals. During the design process, for the selection of constant coefficients of the controller, the PSO (particle swarm optimizer) is applied. Moreover, in order to obtain better dynamic properties and improve work in real conditions, one model based on the ADALINE (adaptive linear neuron) is introduced into the structure. Details of the algorithm used for the weights’ adaptation are presented (including stability analysis) to perform the shaft torque signal filtering. The effectiveness of the proposed approach is examined through simulation and experimental studies.

scholarly journals Application of Machine Learning Algorithms for Solving Problems in the Oil and Gas Sector

2021 ◽  
Vol 19 (3) ◽  
pp. 55-64
Author(s):  
K. N. Maiorov ◽  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Reinforcement Learning ◽  
Machine Learning Algorithms ◽  
Design Stage ◽  
Learning Methods ◽  
Field Development ◽  
Advantages And Disadvantages ◽  
Machine Learning Methods ◽  
The Neural Network

The paper examines the life cycle of field development, analyzes the processes of the field development design stage for the application of machine learning methods. For each process, relevant problems are highlighted, existing solutions based on machine learning methods, ideas and problems are proposed that could be effectively solved by machine learning methods. For the main part of the processes, examples of solutions are briefly described; the advantages and disadvantages of the approaches are identified. The most common solution method is feed-forward neural networks. Subject to preliminary normalization of the input data, this is the most versatile algorithm for regression and classification problems. However, in the problem of selecting wells for hydraulic fracturing, a whole ensemble of machine learning models was used, where, in addition to a neural network, there was a random forest, gradient boosting and linear regression. For the problem of optimizing the placement of a grid of oil wells, the disadvantages of existing solutions based on a neural network and a simple reinforcement learning approach based on Markov decision-making process are identified. A deep reinforcement learning algorithm called Alpha Zero is proposed, which has previously shown significant results in the role of artificial intelligence for games. This algorithm is a decision tree search that directs the neural network: only those branches that have received the best estimates from the neural network are considered more thoroughly. The paper highlights the similarities between the tasks for which Alpha Zero was previously used, and the task of optimizing the placement of a grid of oil producing wells. Conclusions are made about the possibility of using and modifying the algorithm of the optimization problem being solved. Аn approach is proposed to take into account symmetric states in a Monte Carlo tree to reduce the number of required simulations.

Analysis of Symbol Generation and Integration in a Unified Model Based on a Neural Network

2005 ◽  
Vol 9 (3) ◽  
pp. 297-303
Author(s):  
Yukari Yamauchi ◽  
◽  
Shun'ichi Tano ◽  
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Simple Model ◽  
Unified Model ◽  
Symbolic Processing ◽  
Advantages And Disadvantages ◽  
Intelligent Processing ◽  
Knowledge Based ◽  
The Neural Network ◽  
Trained Neural Network

The computational (numerical information) and symbolic (knowledge-based) processing used in intelligent processing has advantages and disadvantages. A simple model integrating symbols into a neural network was proposed as a first step toward fusing computational and symbolic processing. To verify the effectiveness of this model, we first analyze the trained neural network and generate symbols manually. Then we discuss generation methods that are able to discover effective symbols during training of the neural network. We evaluated these through simulations of reinforcement learning in simple football games. Results indicate that the integration of symbols into the neural network improved the performance of player agents.

scholarly journals Reinforcement Learning-Based Satellite Attitude Stabilization Method for Non-Cooperative Target Capturing

Sensors ◽  
2018 ◽  
Vol 18 (12) ◽  
pp. 4331 ◽  
Author(s):  
Zhong Ma ◽  
Yuejiao Wang ◽  
Yidai Yang ◽  
Zhuping Wang ◽  
Lei Tang ◽  
...  
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Attitude Control ◽  
Dynamics Simulation ◽  
Continuous Control ◽  
Simulation Environment ◽  
Control Torque ◽  
The Neural Network

When a satellite performs complex tasks such as discarding a payload or capturing a non-cooperative target, it will encounter sudden changes in the attitude and mass parameters, causing unstable flying and rolling of the satellite. In such circumstances, the change of the movement and mass characteristics are unpredictable. Thus, the traditional attitude control methods are unable to stabilize the satellite since they are dependent on the mass parameters of the controlled object. In this paper, we proposed a reinforcement learning method to re-stabilize the attitude of a satellite under such circumstances. Specifically, we discretize the continuous control torque, and build a neural network model that can output the discretized control torque to control the satellite. A dynamics simulation environment of the satellite is built, and the deep Q Network algorithm is then performed to train the neural network in this simulation environment. The reward of the training is the stabilization of the satellite. Simulation experiments illustrate that, with the iteration of training progresses, the neural network model gradually learned to re-stabilize the attitude of a satellite after unknown disturbance. As a contrast, the traditional PD (Proportion Differential) controller was unable to re-stabilize the satellite due to its dependence on the mass parameters. The proposed method adopts self-learning to control satellite attitudes, shows considerable intelligence and certain universality, and has a strong application potential for future intelligent control of satellites performing complex space tasks.

scholarly journals Reinforcement Learning for Improving the Accuracy of PM2.5 Pollution Forecast Under the Neural Network Framework

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 9864-9874
Author(s):  
Shuo-Wen Chang ◽  
Chung-Ling Chang ◽  
Long-Tin Li ◽  
Shih-Wei Liao
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
The Neural Network ◽  
Pollution Forecast

scholarly journals On the use of neural regulators

2021 ◽  
pp. 53-63
Author(s):  
Alexsander Voevoda ◽  
◽  
Dmitry Romannikov ◽  
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Control Systems ◽  
State Vector ◽  
Neural Controller ◽  
Discrete Form ◽  
Learning Methods ◽  
Data Set ◽  
The Neural Network ◽  
Input Error

The application of neural networks for the synthesis of control systems is considered. Examples of synthesis of control systems using methods of reinforcement learning, in which the state vector is involved, are given. And the synthesis of a neural controller for objects with an inaccessible state vector is discussed: 1) a variant using a neural network with recurrent feedbacks; 2) a variant using the input error vector, where each error (except for the first one) enters the input of the neural network passing through the delay element. The disadvantages of the first method include the fact that for such a structure of a neural network it is not possible to apply existing learning methods with confirmation and for training it is required to use a data set obtained, for example, from a previously calculated linear controller. The structure of the neural network used in the second option allows the application of reinforcement learning methods, but the article provides a statement and its proof that for the synthesis of a control system for objects with three or more integrators, a neural network without recurrent connections cannot be used. The application of the above structures is given on examples of the synthesis of control systems for objects 1/s2 and 1/s3 presented in a discrete form.

Estimation of Dynamic Properties of Sand Using Artificial Neural Networks

1996 ◽  
Vol 1526 (1) ◽  
pp. 1-5
Author(s):  
S. H. Ni ◽  
C. H. Juang ◽  
P. C. Lu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Shear Modulus ◽  
Dynamic Properties ◽  
Damping Ratio ◽  
Soil Parameters ◽  
Overburden Pressure ◽  
The Neural Network ◽  
Artificial Neural

Dynamic properties of soils are usually determined by time-consuming laboratory tests. This study presents a method for estimating dynamic soil parameters using artificial neural networks. A simple feedforward neural network with back-propagation training algorithm is used. The neural network is trained with actual laboratory data, which consists of six input variables. They are the standard penetration test value, the void ratio, the unit weight, the water content, the effective overburden pressure, and the mean effective confining pressure. The output layer consists of a single neuron, representing shear modulus or damping ratio. Results of the neural network training and testing show that predictions of shear modulus by the neural network approach is reliable although it is less successful in predicting damping ratio.

Intelligent scheduling using a neural network model in conjunction with reinforcement learning

2005 ◽  
Vol 219 (2) ◽  
pp. 229-235
Author(s):  
C J Fourie
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Reinforcement Learning ◽  
Learning From Experience ◽  
Scheduling System ◽  
Simulated Environment ◽  
The Neural Network ◽  
Learning Techniques ◽  
Proposed Model ◽  
Intelligent Scheduling

This paper describes the use of an artificial neural network in conjunction with reinforcement learning techniques to develop an intelligent scheduling system that is capable of learning from experience. In a simulated environment the model controls a mobile robot that transports material to machines. States of ‘happiness’ are defined for each machine, which are the inputs to the neural network. The output of the neural network is the decision on which machine to service next. After every decision, a critic evaluates the decision and a teacher ‘rewards’ the network to encourage good decisions and discourage bad decisions. From the results obtained, it is concluded that the proposed model is capable of learning from past experience and thereby improving the intelligence of the system.

scholarly journals A Parametric Study of a Deep Reinforcement Learning Control System Applied to the Swing-Up Problem of the Cart-Pole

2020 ◽  
Vol 10 (24) ◽  
pp. 9013
Author(s):  
Camilo Andrés Manrique Escobar ◽  
Carmine Maria Pappalardo ◽  
Domenico Guida
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Dry Friction ◽  
Network Architecture ◽  
Control Policy ◽  
Neural Network Architecture ◽  
Learning Capability ◽  
Friction Forces ◽  
The Neural Network ◽  
Pole System

In this investigation, the nonlinear swing-up problem associated with the cart-pole system modeled as a multibody dynamical system is solved by developing a deep Reinforcement Learning (RL) controller. Furthermore, the sensitivity analysis of the deep RL controller applied to the cart-pole swing-up problem is carried out. To this end, the influence of modifying the physical properties of the system and the presence of dry friction forces are analyzed employing the cumulative reward during the task. Extreme limits for the modifications of the parameters are determined to prove that the neural network architecture employed in this work features enough learning capability to handle the task under modifications as high as 90% on the pendulum mass, as well as a 100% increment on the cart mass. As expected, the presence of dry friction greatly affects the performance of the controller. However, a post-training of the agent in the modified environment takes only thirty-nine episodes to find the optimal control policy, resulting in a promising path for further developments of robust controllers.

Sign in / Sign up

Export Citation Format

Share Document