A Neurocomputational Model of Dopamine and Prefrontal–Striatal Interactions during Multicue Category Learning by Parkinson Patients

2011 ◽  
Vol 23 (1) ◽  
pp. 151-167 ◽  
Author(s):  
Ahmed A. Moustafa ◽  
Mark A. Gluck
Keyword(s):  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Category Learning ◽  
Weather Prediction ◽  
Critical Role ◽  
Neural Mechanism ◽  
Competitive Dynamics ◽  
Stimulus Selection ◽  
Striatal Neurons ◽  
Attentional Learning

Most existing models of dopamine and learning in Parkinson disease (PD) focus on simulating the role of basal ganglia dopamine in reinforcement learning. Much data argue, however, for a critical role for prefrontal cortex (PFC) dopamine in stimulus selection in attentional learning. Here, we present a new computational model that simulates performance in multicue category learning, such as the “weather prediction” task. The model addresses how PD and dopamine medications affect stimulus selection processes, which mediate reinforcement learning. In this model, PFC dopamine is key for attentional learning, whereas basal ganglia dopamine, consistent with other models, is key for reinforcement and motor learning. The model assumes that competitive dynamics among PFC neurons is the neural mechanism underlying stimulus selection with limited attentional resources, whereas competitive dynamics among striatal neurons is the neural mechanism underlying action selection. According to our model, PD is associated with decreased phasic and tonic dopamine levels in both PFC and basal ganglia. We assume that dopamine medications increase dopamine levels in both the basal ganglia and PFC, which, in turn, increase tonic dopamine levels but decrease the magnitude of phasic dopamine signaling in these brain structures. Increase of tonic dopamine levels in the simulated PFC enhances attentional shifting performance. The model provides a mechanistic account for several phenomena, including (a) medicated PD patients are more impaired at multicue probabilistic category learning than unmedicated patients and (b) medicated PD patients opt out of reversal when there are alternative and redundant cue dimensions.

scholarly journals Theory of reinforcement learning and motivation in the basal ganglia

2017 ◽  
Author(s):  
Rafal Bogacz
Keyword(s):  
Synaptic Plasticity ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Dopaminergic Neurons ◽  
Neural Circuits ◽  
Negative Consequences ◽  
Striatal Neurons ◽  
Motivational State ◽  
Level Of Activity ◽  
Dopaminergic Modulation

AbstractThis paper proposes how the neural circuits in vertebrates select actions on the basis of past experience and the current motivational state. According to the presented theory, the basal ganglia evaluate the utility of considered actions by combining the positive consequences (e.g. nutrition) scaled by the motivational state (e.g. hunger) with the negative consequences (e.g. effort). The theory suggests how the basal ganglia compute utility by combining the positive and negative consequences encoded in the synaptic weights of striatal Go and No-Go neurons, and the motivational state carried by neuromodulators including dopamine. Furthermore, the theory suggests how the striatal neurons to learn separately about consequences of actions, and how the dopaminergic neurons themselves learn what level of activity they need to produce to optimize behaviour. The theory accounts for the effects of dopaminergic modulation on behaviour, patterns of synaptic plasticity in striatum, and responses of dopaminergic neurons in diverse situations.

scholarly journals Striatal dynamics explain duration judgments

2015 ◽  
Author(s):  
Thiago S. Gouvêa ◽  
Tiago Monteiro ◽  
Asma Motiwala ◽  
Sofia Soares ◽  
Christian K. Machens ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Interval Timing ◽  
Time Dependent ◽  
Striatal Neurons ◽  
Input Structure ◽  
State Changes ◽  
Psychophysical Task

The striatum is an input structure of the basal ganglia implicated in several time-dependent functions including reinforcement learning, decision making, and interval timing. To determine whether striatal ensembles drive subjects' judgments of duration, we manipulated and recorded from striatal neurons in rats performing a duration categorization psychophysical task. We found that the dynamics of striatal neurons predicted duration judgments, and that simultaneously recorded ensembles could judge duration as well as the animal. Furthermore, striatal neurons were necessary for duration judgments, as muscimol infusions produced a specific impairment in animals' duration sensitivity. Lastly, we show that time as encoded by striatal populations ran faster or slower when rats judged a duration as longer or shorter, respectively. These results demonstrate that the speed with which striatal population state changes supports the fundamental ability of animals to judge the passage of time.

scholarly journals Striatal dynamics explain duration judgments

eLife ◽  
2015 ◽  
Vol 4 ◽  
Author(s):  
Thiago S Gouvêa ◽  
Tiago Monteiro ◽  
Asma Motiwala ◽  
Sofia Soares ◽  
Christian Machens ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Interval Timing ◽  
Time Dependent ◽  
Striatal Neurons ◽  
Input Structure ◽  
State Changes ◽  
Psychophysical Task

The striatum is an input structure of the basal ganglia implicated in several time-dependent functions including reinforcement learning, decision making, and interval timing. To determine whether striatal ensembles drive subjects' judgments of duration, we manipulated and recorded from striatal neurons in rats performing a duration categorization psychophysical task. We found that the dynamics of striatal neurons predicted duration judgments, and that simultaneously recorded ensembles could judge duration as well as the animal. Furthermore, striatal neurons were necessary for duration judgments, as muscimol infusions produced a specific impairment in animals' duration sensitivity. Lastly, we show that time as encoded by striatal populations ran faster or slower when rats judged a duration as longer or shorter, respectively. These results demonstrate that the speed with which striatal population state changes supports the fundamental ability of animals to judge the passage of time.

scholarly journals The Lamprey Forebrain – Evolutionary Implications

2021 ◽  
pp. 1-16
Author(s):  
Shreyas M. Suryanarayana ◽  
Juan Pérez-Fernández ◽  
Brita Robertson ◽  
Sten Grillner
Keyword(s):  
Basal Ganglia ◽  
Sensory Integration ◽  
Critical Role ◽  
Detailed Comparison ◽  
Common Ancestry ◽  
Dopamine System ◽  
Cortical Areas ◽  
Neural Processes ◽  
Living Group ◽  
Selection Of

The forebrain plays a critical role in a broad range of neural processes encompassing sensory integration and initiation/selection of behaviour. The forebrain functions through an interaction between different cortical areas, the thalamus, the basal ganglia with the dopamine system, and the habenulae. The ambition here is to compare the mammalian forebrain with that of the lamprey representing the oldest now living group of vertebrates, by a review of earlier studies. We show that the lamprey dorsal pallium has a motor, a somatosensory, and a visual area with retinotopic representation. The lamprey pallium was previously thought to be largely olfactory. There is also a detailed similarity between the lamprey and mammals with regard to other forebrain structures like the basal ganglia in which the general organisation, connectivity, transmitters and their receptors, neuropeptides, and expression of ion channels are virtually identical. These initially unexpected results allow for the possibility that many aspects of the basic design of the vertebrate forebrain had evolved before the lamprey diverged from the evolutionary line leading to mammals. Based on a detailed comparison between the mammalian forebrain and that of the lamprey and with due consideration of data from other vertebrate groups, we propose a compelling account of a pan-vertebrate schema for basic forebrain structures, suggesting a common ancestry of over half a billion years of vertebrate evolution.

scholarly journals Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach

Energies ◽  
2021 ◽  
Vol 14 (9) ◽  
pp. 2700
Author(s):  
Grace Muriithi ◽  
Sunetra Chowdhury
Keyword(s):  
Renewable Energy ◽  
Reinforcement Learning ◽  
Energy Management ◽  
Renewable Energy Sources ◽  
Critical Role ◽  
Winter Season ◽  
Learning Approach ◽  
Solar Pv ◽  
Optimal Energy ◽  
Optimal Energy Management

In the near future, microgrids will become more prevalent as they play a critical role in integrating distributed renewable energy resources into the main grid. Nevertheless, renewable energy sources, such as solar and wind energy can be extremely volatile as they are weather dependent. These resources coupled with demand can lead to random variations on both the generation and load sides, thus complicating optimal energy management. In this article, a reinforcement learning approach has been proposed to deal with this non-stationary scenario, in which the energy management system (EMS) is modelled as a Markov decision process (MDP). A novel modification of the control problem has been presented that improves the use of energy stored in the battery such that the dynamic demand is not subjected to future high grid tariffs. A comprehensive reward function has also been developed which decreases infeasible action explorations thus improving the performance of the data-driven technique. A Q-learning algorithm is then proposed to minimize the operational cost of the microgrid under unknown future information. To assess the performance of the proposed EMS, a comparison study between a trading EMS model and a non-trading case is performed using a typical commercial load curve and PV profile over a 24-h horizon. Numerical simulation results indicate that the agent learns to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility and battery wear cost) in all the studied cases. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one was found to decrease costs by 4.033% in summer season and 2.199% in winter season.

scholarly journals Corticostriatal Plastic Changes in Experimental L-DOPA-Induced Dyskinesia

2012 ◽  
Vol 2012 ◽  
pp. 1-10 ◽  
Author(s):  
Veronica Ghiglieri ◽  
Vincenza Bagetta ◽  
Valentina Pendolino ◽  
Barbara Picconi ◽  
Paolo Calabresi
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Basal Ganglia ◽  
Experimental Models ◽  
Striatal Neurons ◽  
Da Receptors ◽  
Electrophysiological Studies ◽  
Plastic Changes ◽  
Stimulation Of

In Parkinson’s disease (PD), alteration of dopamine- (DA-) dependent striatal functions and pulsatile stimulation of DA receptors caused by the discontinuous administration of levodopa (L-DOPA) lead to a complex cascade of events affecting the postsynaptic striatal neurons that might account for the appearance of L-DOPA-induced dyskinesia (LID). Experimental models of LID have been widely used and extensively characterized in rodents and electrophysiological studies provided remarkable insights into the inner mechanisms underlying L-DOPA-induced corticostriatal plastic changes. Here we provide an overview of recent findings that represent a further step into the comprehension of mechanisms underlying maladaptive changes of basal ganglia functions in response to L-DOPA and associated to development of LID.

scholarly journals A selective projection from the subthalamic nucleus to parvalbumin-expressing interneurons of the striatum

2020 ◽  
Author(s):  
Krishnakanth Kondabolu ◽  
Natalie M. Doig ◽  
Olaoluwa Ayeko ◽  
Bakhtawer Khan ◽  
Alexandra Torres ◽  
...  
Keyword(s):  
Basal Ganglia ◽  
Subthalamic Nucleus ◽  
Ex Vivo ◽  
Cell Types ◽  
Projection Neurons ◽  
Striatal Neurons ◽  
Primary Input ◽  
Cholinergic Interneurons ◽  
Axonal Connections ◽  
Excitatory Responses

AbstractThe striatum and subthalamic nucleus (STN) are considered to be the primary input nuclei of the basal ganglia. Projection neurons of both striatum and STN can extensively interact with other basal ganglia nuclei, and there is growing anatomical evidence of direct axonal connections from the STN to striatum. There remains, however, a pressing need to elucidate the organization and impact of these subthalamostriatal projections in the context of the diverse cell types constituting the striatum. To address this, we carried out monosynaptic retrograde tracing from genetically-defined populations of dorsal striatal neurons in adult male and female mice, quantifying the connectivity from STN neurons to spiny projection neurons, GABAergic interneurons, and cholinergic interneurons. In parallel, we used a combination of ex vivo electrophysiology and optogenetics to characterize the responses of a complementary range of dorsal striatal neuron types to activation of STN axons. Our tracing studies showed that the connectivity from STN neurons to striatal parvalbumin-expressing interneurons is significantly higher (~ four-to eight-fold) than that from STN to any of the four other striatal cell types examined. In agreement, our recording experiments showed that parvalbumin-expressing interneurons, but not the other cell types tested, commonly exhibited robust monosynaptic excitatory responses to subthalamostriatal inputs. Taken together, our data collectively demonstrate that the subthalamostriatal projection is highly selective for target cell type. We conclude that glutamatergic STN neurons are positioned to directly and powerfully influence striatal activity dynamics by virtue of their enriched innervation of GABAergic parvalbumin-expressing interneurons.

Sign in / Sign up

Export Citation Format

Share Document