Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning

2001 ◽  
Vol 13 (10) ◽  
pp. 2221-2237 ◽  
Author(s):  
Rajesh P. N. Rao ◽  
Terrence J. Sejnowski
Keyword(s):  
Cortical Neuron ◽  
Action Potentials ◽  
Learning Rule ◽  
Spike Timing ◽  
Biophysical Model ◽  
Excitatory Synapses ◽  
Temporal Difference ◽  
Temporal Difference Learning ◽  
Neocortical Neurons ◽  
Hebbian Plasticity

A spike-timing-dependent Hebbian mechanism governs the plasticity of recurrent excitatory synapses in the neocortex: synapses that are activated a few milliseconds before a postsynaptic spike are potentiated, while those that are activated a few milliseconds after are depressed. We show that such a mechanism can implement a form of temporal difference learning for prediction of input sequences. Using a biophysical model of a cortical neuron, we show that a temporal difference rule used in conjunction with dendritic backpropagating action potentials reproduces the temporally asymmetric window of Hebbian plasticity observed physiologically. Furthermore, the size and shape of the window vary with the distance of the synapse from the soma. Using a simple example, we show how a spike-timing-based temporal difference learning rule can allow a network of neocortical neurons to predict an input a few milliseconds before the input's expected arrival.

scholarly journals Effects of memristive synapse radiation interactions on learning in spiking neural networks

2021 ◽  
Vol 3 (5) ◽  
Author(s):  
Sumedha Gandharava Dahl ◽  
Robert C. Ivans ◽  
Kurtis D. Cantley
Keyword(s):  
Action Potentials ◽  
Radiation Effects ◽  
Synaptic Weight ◽  
Learning Rule ◽  
Spike Timing ◽  
Spiking Neural Network ◽  
Synaptic Action ◽  
Drift Model ◽  
Spatio Temporal ◽  
Post Synaptic Neuron

AbstractThis study uses advanced modeling and simulation to explore the effects of external events such as radiation interactions on the synaptic devices in an electronic spiking neural network. Specifically, the networks are trained using the spike-timing-dependent plasticity (STDP) learning rule to recognize spatio-temporal patterns (STPs) representing 25 and 100-pixel characters. Memristive synapses based on a TiO2 non-linear drift model designed in Verilog-A are utilized, with STDP learning behavior achieved through bi-phasic pre- and post-synaptic action potentials. The models are modified to include experimentally observed state-altering and ionizing radiation effects on the device. It is found that radiation interactions tend to make the connection between afferents stronger by increasing the conductance of synapses overall, subsequently distorting the STDP learning curve. In the absence of consistent STPs, these effects accumulate over time and make the synaptic weight evolutions unstable. With STPs at lower flux intensities, the network can recover and relearn with constant training. However, higher flux can overwhelm the leaky integrate-and-fire post-synaptic neuron circuits and reduce stability of the network.

scholarly journals Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation

eLife ◽  
2017 ◽  
Vol 6 ◽  
Author(s):  
Zuzanna Brzosko ◽  
Sara Zannone ◽  
Wolfram Schultz ◽  
Claudia Clopath ◽  
Ole Paulsen
Keyword(s):  
Time Window ◽  
Hippocampal Slices ◽  
Learning Rule ◽  
Spike Timing ◽  
Synaptic Potentiation ◽  
Behavioral Learning ◽  
Hebbian Plasticity ◽  
Subsequent Application ◽  
Behavioral States ◽  
Synaptic Learning

Spike timing-dependent plasticity (STDP) is under neuromodulatory control, which is correlated with distinct behavioral states. Previously, we reported that dopamine, a reward signal, broadens the time window for synaptic potentiation and modulates the outcome of hippocampal STDP even when applied after the plasticity induction protocol (Brzosko et al., 2015). Here, we demonstrate that sequential neuromodulation of STDP by acetylcholine and dopamine offers an efficacious model of reward-based navigation. Specifically, our experimental data in mouse hippocampal slices show that acetylcholine biases STDP toward synaptic depression, whilst subsequent application of dopamine converts this depression into potentiation. Incorporating this bidirectional neuromodulation-enabled correlational synaptic learning rule into a computational model yields effective navigation toward changing reward locations, as in natural foraging behavior. Thus, temporally sequenced neuromodulation of STDP enables associations to be made between actions and outcomes and also provides a possible mechanism for aligning the time scales of cellular and behavioral learning.

How the Shape of Pre- and Postsynaptic Signals Can Influence STDP: A Biophysical Model

2004 ◽  
Vol 16 (3) ◽  
pp. 595-625 ◽  
Author(s):  
Ausra Saudargiene ◽  
Bernd Porr ◽  
Florentin Wörgötter
Keyword(s):  
Membrane Potential ◽  
Weight Change ◽  
Hebbian Learning ◽  
Dendritic Tree ◽  
Learning Rule ◽  
Long Term Potentiation ◽  
Spike Timing ◽  
Biophysical Model ◽  
Nmda Channel

Spike-timing-dependent plasticity (STDP) is described by long-term potentiation (LTP), when a presynaptic event precedes a postsynaptic event, and by long-term depression (LTD), when the temporal order is reversed. In this article, we present a biophysical model of STDP based on a differential Hebbian learning rule (ISO learning). This rule correlates presynaptically the NMDA channel conductance with the derivative of the membrane potential at the synapse as the postsynaptic signal. The model is able to reproduce the generic STDP weight change characteristic. We find that (1) The actual shape of the weight change curve strongly depends on the NMDA channel characteristics and on the shape of the membrane potential at the synapse. (2) The typical antisymmetrical STDP curve (LTD and LTP) can become similar to a standard Hebbian characteristic (LTP only) without having to change the learning rule. This occurs if the membrane depolarization has a shallow onset and is long lasting. (3) It is known that the membrane potential varies along the dendrite as a result of the active or passive backpropagation of somatic spikes or because of local dendritic processes. As a consequence, our model predicts that learning properties will be different at different locations on the dendritic tree. In conclusion, such site-specific synaptic plasticity would provide a neuron with powerful learning capabilities.

Learning Only When Necessary: Better Memories of Correlated Patterns in Networks with Bounded Synapses

2005 ◽  
Vol 17 (10) ◽  
pp. 2106-2138 ◽  
Author(s):  
Walter Senn ◽  
Stefano Fusi
Keyword(s):  
Learning Rule ◽  
Neural Dynamics ◽  
Excitatory Synapses ◽  
Linear Superposition ◽  
Hebbian Plasticity ◽  
Correlated Patterns ◽  
Random Patterns ◽  
Highly Correlated ◽  
Additional Constraints ◽  
Global Inhibition

Learning in a neuronal network is often thought of as a linear superposition of synaptic modifications induced by individual stimuli. However, since biological synapses are naturally bounded, a linear superposition would cause fast forgetting of previously acquired memories. Here we show that this forgetting can be avoided by introducing additional constraints on the synaptic and neural dynamics. We consider Hebbian plasticity of excitatory synapses. A synapse is modified only if the postsynaptic response does not match the desired output. With this learning rule, the original memory performances with unbounded weights are regained, provided that (1) there is some global inhibition, (2) the learning rate is small, and (3) the neurons can discriminate small differences in the total synaptic input (e.g., by making the neuronal threshold small compared to the total postsynaptic input). We prove in the form of a generalized perceptron convergence theorem that under these constraints, a neuron learns to classify any linearly separable set of patterns, including a wide class of highly correlated random patterns. During the learning process, excitation becomes roughly balanced by inhibition, and the neuron classifies the patterns on the basis of small differences around this balance. The fact that synapses saturate has the additional benefit that nonlinearly separable patterns, such as similar patterns with contradicting outputs, eventually generate a subthreshold response, and therefore silence neurons that cannot provide any information.

scholarly journals A Biophysical Model of Synaptic Plasticity and Metaplasticity Can Account for the Dynamics of the Backward Shift of Hippocampal Place Fields

2008 ◽  
Vol 100 (2) ◽  
pp. 983-992 ◽  
Author(s):  
Xintian Yu ◽  
Harel Z. Shouval ◽  
James J. Knierim
Keyword(s):  
Synaptic Plasticity ◽  
Cortical Plasticity ◽  
Learning Rule ◽  
Spike Timing ◽  
Biophysical Model ◽  
Place Field ◽  
Dependent Plasticity ◽  
Backward Shift ◽  
Field Shift

Hippocampal place cells in the rat undergo experience-dependent changes when the rat runs stereotyped routes. One such change, the backward shift of the place field center of mass, has been linked by previous modeling efforts to spike-timing–dependent plasticity (STDP). However, these models did not account for the termination of the place field shift and they were based on an abstract implementation of STDP that ignores many of the features found in cortical plasticity. Here, instead of the abstract STDP model, we use a calcium-dependent plasticity (CaDP) learning rule that can account for many of the observed properties of cortical plasticity. We use the CaDP learning rule in combination with a model of metaplasticity to simulate place field dynamics. Without any major changes to the parameters of the original model, the present simulations account both for the initial rapid place field shift and for the subsequent slowing down of this shift. These results suggest that the CaDP model captures the essence of a general cortical mechanism of synaptic plasticity, which may underlie numerous forms of synaptic plasticity observed both in vivo and in vitro.

Synaptic Democracy in Active Dendrites

2006 ◽  
Vol 96 (5) ◽  
pp. 2307-2318 ◽  
Author(s):  
Clifton C. Rumsey ◽  
L. F. Abbott
Keyword(s):  
Action Potentials ◽  
Dendritic Tree ◽  
Pyramidal Cells ◽  
Spike Timing ◽  
Dominant Factor ◽  
Dependent Plasticity ◽  
Ca1 Pyramidal Cells ◽  
Active Dendrites ◽  
Hebbian Plasticity ◽  
The Impact

Given the extensive attenuation that can occur along dendritic cables, location within the dendritic tree might appear to be a dominant factor in determining the impact of a synapse on the postsynaptic response. By this reasoning, distal synapses should have a smaller effect than proximal ones. However, experimental evidence from several types of neurons, such as CA1 pyramidal cells, indicates that a compensatory strengthening of synapses counteracts the effect of location on synaptic efficacy. A form of spike-timing-dependent plasticity (STDP), called anti-STDP, combined with non-Hebbian activity-dependent plasticity can account for the equalization of synaptic efficacies. This result, obtained originally in models with unbranched passive cables, also arises in multi-compartment models with branched and active dendrites that feature backpropagating action potentials, including models with CA1 pyramidal morphologies. Additionally, when dendrites support the local generation of action potentials, anti-STDP prevents runaway dendritic spiking and locally balances the numbers of dendritic and backpropagating action potentials. Thus in multiple ways, anti-STDP eliminates the location dependence of synapses and allows Hebbian plasticity to operate in a more “democratic” manner.

Sign in / Sign up

Export Citation Format

Share Document