scholarly journals Retroactive modulation of spike timing-dependent plasticity by dopamine

eLife ◽  
2015 ◽  
Vol 4 ◽  
Author(s):  
Zuzanna Brzosko ◽  
Wolfram Schultz ◽  
Ole Paulsen
Keyword(s):  
Reinforcement Learning ◽  
Nmda Receptors ◽  
Positive Reinforcement ◽  
Spike Timing ◽  
Learning Models ◽  
Cellular Mechanisms ◽  
Dependent Plasticity ◽  
Eligibility Trace ◽  
Plausible Mechanism ◽  
Reinforcement Learning Models

Most reinforcement learning models assume that the reward signal arrives after the activity that led to the reward, placing constraints on the possible underlying cellular mechanisms. Here we show that dopamine, a positive reinforcement signal, can retroactively convert hippocampal timing-dependent synaptic depression into potentiation. This effect requires functional NMDA receptors and is mediated in part through the activation of the cAMP/PKA cascade. Collectively, our results support the idea that reward-related signaling can act on a pre-established synaptic eligibility trace, thereby associating specific experiences with behaviorally distant, rewarding outcomes. This finding identifies a biologically plausible mechanism for solving the ‘distal reward problem’.

Role of AMPA and NMDA Receptors and Back-Propagating Action Potentials in Spike Timing–Dependent Plasticity

2010 ◽  
Vol 103 (1) ◽  
pp. 47-54 ◽  
Author(s):  
Marco Fuenzalida ◽  
David Fernández de Sevilla ◽  
Alejandro Couve ◽  
Washington Buño
Keyword(s):  
Nmda Receptors ◽  
Pyramidal Neurons ◽  
Long Term Potentiation ◽  
Receptor Activation ◽  
Spike Timing ◽  
Spike Timing Dependent Plasticity ◽  
Cellular Mechanisms ◽  
Dependent Plasticity ◽  
Term Potentiation

The cellular mechanisms that mediate spike timing–dependent plasticity (STDP) are largely unknown. We studied in vitro in CA1 pyramidal neurons the contribution of AMPA and N-methyl-d-aspartate (NMDA) components of Schaffer collateral (SC) excitatory postsynaptic potentials (EPSPs; EPSPAMPA and EPSPNMDA) and of the back-propagating action potential (BAP) to the long-term potentiation (LTP) induced by a STDP protocol that consisted in pairing an EPSP and a BAP. Transient blockade of EPSPAMPA with 7-nitro-2,3-dioxo-1,4-dihydroquinoxaline-6-carbonitrile (CNQX) during the STDP protocol prevented LTP. Contrastingly LTP was induced under transient inhibition of EPSPAMPA by combining SC stimulation, an imposed EPSPAMPA-like depolarization, and BAP or by coupling the EPSPNMDA evoked under sustained depolarization (approximately −40 mV) and BAP. In Mg2+-free solution EPSPNMDA and BAP also produced LTP. Suppression of EPSPNMDA or BAP always prevented LTP. Thus activation of NMDA receptors and BAPs are needed but not sufficient because AMPA receptor activation is also obligatory for STDP. However, a transient depolarization of another origin that unblocks NMDA receptors and a BAP may also trigger LTP.

Bayes factors for reinforcement-learning models of the Iowa gambling task.

Decision ◽  
2016 ◽  
Vol 3 (2) ◽  
pp. 115-131 ◽  
Author(s):  
Helen Steingroever ◽  
Ruud Wetzels ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Reinforcement Learning ◽  
Iowa Gambling Task ◽  
Bayes Factors ◽  
Gambling Task ◽  
Learning Models ◽  
Reinforcement Learning Models

scholarly journals Test-retest reliability of canonical reinforcement learning models

2019 ◽  
Author(s):  
Laura Weidinger ◽  
Andrea Gradassi ◽  
Lucas Molleman ◽  
Wouter van den Bos
Keyword(s):  
Reinforcement Learning ◽  
Learning Models ◽  
Retest Reliability ◽  
Reinforcement Learning Models ◽  
Test Retest Reliability

scholarly journals Balanced networks under spike-time dependent plasticity

2021 ◽  
Vol 17 (5) ◽  
pp. e1008958
Author(s):  
Alan Eric Akil ◽  
Robert Rosenbaum ◽  
Krešimir Josić
Keyword(s):  
Spike Timing ◽  
Recurrent Networks ◽  
Spike Time ◽  
Weight Changes ◽  
Initial State ◽  
Dependent Plasticity ◽  
Plausible Mechanism ◽  
Spike Time Dependent Plasticity ◽  
Induced Changes ◽  
Balanced Networks

The dynamics of local cortical networks are irregular, but correlated. Dynamic excitatory–inhibitory balance is a plausible mechanism that generates such irregular activity, but it remains unclear how balance is achieved and maintained in plastic neural networks. In particular, it is not fully understood how plasticity induced changes in the network affect balance, and in turn, how correlated, balanced activity impacts learning. How do the dynamics of balanced networks change under different plasticity rules? How does correlated spiking activity in recurrent networks change the evolution of weights, their eventual magnitude, and structure across the network? To address these questions, we develop a theory of spike–timing dependent plasticity in balanced networks. We show that balance can be attained and maintained under plasticity–induced weight changes. We find that correlations in the input mildly affect the evolution of synaptic weights. Under certain plasticity rules, we find an emergence of correlations between firing rates and synaptic weights. Under these rules, synaptic weights converge to a stable manifold in weight space with their final configuration dependent on the initial state of the network. Lastly, we show that our framework can also describe the dynamics of plastic balanced networks when subsets of neurons receive targeted optogenetic input.

scholarly journals Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity

2007 ◽  
Vol 19 (6) ◽  
pp. 1468-1502 ◽  
Author(s):  
Răzvan V. Florian
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Neural Model ◽  
Spike Timing ◽  
Spike Response ◽  
Learning Rules ◽  
Xor Problem ◽  
Eligibility Trace ◽  
Spike Response Model ◽  
Intrinsic Plasticity

The persistent modification of synaptic efficacy as a function of the relative timing of pre- and postsynaptic spikes is a phenomenon known as spike-timing-dependent plasticity (STDP). Here we show that the modulation of STDP by a global reward signal leads to reinforcement learning. We first derive analytically learning rules involving reward-modulated spike-timing-dependent synaptic and intrinsic plasticity, by applying a reinforcement learning algorithm to the stochastic spike response model of spiking neurons. These rules have several features common to plasticity mechanisms experimentally found in the brain. We then demonstrate in simulations of networks of integrate-and-fire neurons the efficacy of two simple learning rules involving modulated STDP. One rule is a direct extension of the standard STDP model (modulated STDP), and the other one involves an eligibility trace stored at each synapse that keeps a decaying memory of the relationships between the recent pairs of pre- and postsynaptic spike pairs (modulated STDP with eligibility trace). This latter rule permits learning even if the reward signal is delayed. The proposed rules are able to solve the XOR problem with both rate coded and temporally coded input and to learn a target output firing-rate pattern. These learning rules are biologically plausible, may be used for training generic artificial spiking neural networks, regardless of the neural model used, and suggest the experimental investigation in animals of the existence of reward-modulated STDP.

Sign in / Sign up

Export Citation Format

Share Document