scholarly journals A neural network model for the orbitofrontal cortex and task space acquisition during reinforcement learning

2018 ◽  
Vol 14 (1) ◽  
pp. e1005925 ◽  
Author(s):  
Zhewei Zhang ◽  
Zhenbo Cheng ◽  
Zhongqiao Lin ◽  
Chechang Nie ◽  
Tianming Yang
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Orbitofrontal Cortex ◽  
Task Space ◽  
Space Acquisition ◽  
And Task

scholarly journals A neural network model for the orbitofrontal cortex and task space acquisition during reinforcement learning

2017 ◽  
Author(s):  
Zhewei Zhang ◽  
Zhenbo Cheng ◽  
Zhongqiao Lin ◽  
Chechang Nie ◽  
Tianming Yang
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
State Space ◽  
Network Model ◽  
Neural Network Model ◽  
Orbitofrontal Cortex ◽  
Activity Patterns ◽  
Neural Mechanism ◽  
Theoretical Explanation ◽  
Dynamic Activity

AbstractReinforcement learning has been widely used in explaining animal behavior. In reinforcement learning, the agent learns the value of the states in the task, collectively constituting the task state space, and use the knowledge to choose actions and acquire desired outcomes. It has been proposed that the orbitofrontal cortex (OFC) encodes the task state space during reinforcement learning. However, it is not well understood how the OFC acquires and stores task state information. Here, we propose a neural network model based on reservoir computing. Reservoir networks exhibit heterogeneous and dynamic activity patterns that are suitable to encode task states. The information can be extracted by a linear readout trained with reinforcement learning. We demonstrate how the network acquires and stores the task structures. The network exhibits reinforcement learning behavior and its aspects resemble experimental findings of the OFC. Our study provides a theoretical explanation of how the OFC may contribute to reinforcement learning and a new approach to understanding the neural mechanism underlying reinforcement learning.

scholarly journals Reinforcement Learning-Based Satellite Attitude Stabilization Method for Non-Cooperative Target Capturing

Sensors ◽  
2018 ◽  
Vol 18 (12) ◽  
pp. 4331 ◽  
Author(s):  
Zhong Ma ◽  
Yuejiao Wang ◽  
Yidai Yang ◽  
Zhuping Wang ◽  
Lei Tang ◽  
...  
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Attitude Control ◽  
Dynamics Simulation ◽  
Continuous Control ◽  
Simulation Environment ◽  
Control Torque ◽  
The Neural Network

When a satellite performs complex tasks such as discarding a payload or capturing a non-cooperative target, it will encounter sudden changes in the attitude and mass parameters, causing unstable flying and rolling of the satellite. In such circumstances, the change of the movement and mass characteristics are unpredictable. Thus, the traditional attitude control methods are unable to stabilize the satellite since they are dependent on the mass parameters of the controlled object. In this paper, we proposed a reinforcement learning method to re-stabilize the attitude of a satellite under such circumstances. Specifically, we discretize the continuous control torque, and build a neural network model that can output the discretized control torque to control the satellite. A dynamics simulation environment of the satellite is built, and the deep Q Network algorithm is then performed to train the neural network in this simulation environment. The reward of the training is the stabilization of the satellite. Simulation experiments illustrate that, with the iteration of training progresses, the neural network model gradually learned to re-stabilize the attitude of a satellite after unknown disturbance. As a contrast, the traditional PD (Proportion Differential) controller was unable to re-stabilize the satellite due to its dependence on the mass parameters. The proposed method adopts self-learning to control satellite attitudes, shows considerable intelligence and certain universality, and has a strong application potential for future intelligent control of satellites performing complex space tasks.

scholarly journals A Recurrent Neural Network Model for Flexible and Adaptive Decision Making based on Sequence Learning

2019 ◽  
Author(s):  
Zhewei Zhang ◽  
Huzi Cheng ◽  
Tianming Yang
Keyword(s):  
Neural Network ◽  
Decision Making ◽  
Reinforcement Learning ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Adaptive Behavior ◽  
Neural Network Model ◽  
Sequence Learning ◽  
Sensory Inputs ◽  
The Brain

AbstractThe brain makes flexible and adaptive responses in the complicated and ever-changing environment for the organism’s survival. To achieve this, the brain needs to choose appropriate actions flexibly in response to sensory inputs. Moreover, the brain also has to understand how its actions affect future sensory inputs and what reward outcomes should be expected, and adapts its behavior based on the actual outcomes. A modeling approach that takes into account of the combined contingencies between sensory inputs, actions, and reward outcomes may be the key to understanding the underlying neural computation. Here, we train a recurrent neural network model based on sequence learning to predict future events based on the past event sequences that combine sensory, action, and reward events. We use four exemplary tasks that have been used in previous animal and human experiments to study different aspects of decision making and learning. We first show that the model reproduces the animals’ choice and reaction time pattern in a probabilistic reasoning task, and its units’ activities mimics the classical findings of the ramping pattern of the parietal neurons that reflects the evidence accumulation process during decision making. We further demonstrate that the model carries out Bayesian inference and may support meta-cognition such as confidence with additional tasks. Finally, we show how the network model achieves adaptive behavior with an approach distinct from reinforcement learning. Our work pieces together many experimental findings in decision making and reinforcement learning and provides a unified framework for the flexible and adaptive behavior of the brain.

Sign in / Sign up

Export Citation Format

Share Document