Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation

Spike timing-dependent plasticity (STDP) is under neuromodulatory control, which is correlated with distinct behavioral states. Previously, we reported that dopamine, a reward signal, broadens the time window for synaptic potentiation and modulates the outcome of hippocampal STDP even when applied after the plasticity induction protocol (Brzosko et al., 2015). Here, we demonstrate that sequential neuromodulation of STDP by acetylcholine and dopamine offers an efficacious model of reward-based navigation. Specifically, our experimental data in mouse hippocampal slices show that acetylcholine biases STDP toward synaptic depression, whilst subsequent application of dopamine converts this depression into potentiation. Incorporating this bidirectional neuromodulation-enabled correlational synaptic learning rule into a computational model yields effective navigation toward changing reward locations, as in natural foraging behavior. Thus, temporally sequenced neuromodulation of STDP enables associations to be made between actions and outcomes and also provides a possible mechanism for aligning the time scales of cellular and behavioral learning.

Download Full-text

Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning

Neural Computation ◽

10.1162/089976601750541787 ◽

2001 ◽

Vol 13 (10) ◽

pp. 2221-2237 ◽

Cited By ~ 118

Author(s):

Rajesh P. N. Rao ◽

Terrence J. Sejnowski

Keyword(s):

Cortical Neuron ◽

Action Potentials ◽

Learning Rule ◽

Spike Timing ◽

Biophysical Model ◽

Excitatory Synapses ◽

Temporal Difference ◽

Temporal Difference Learning ◽

Neocortical Neurons ◽

Hebbian Plasticity

A spike-timing-dependent Hebbian mechanism governs the plasticity of recurrent excitatory synapses in the neocortex: synapses that are activated a few milliseconds before a postsynaptic spike are potentiated, while those that are activated a few milliseconds after are depressed. We show that such a mechanism can implement a form of temporal difference learning for prediction of input sequences. Using a biophysical model of a cortical neuron, we show that a temporal difference rule used in conjunction with dendritic backpropagating action potentials reproduces the temporally asymmetric window of Hebbian plasticity observed physiologically. Furthermore, the size and shape of the window vary with the distance of the synapse from the soma. Using a simple example, we show how a spike-timing-based temporal difference learning rule can allow a network of neocortical neurons to predict an input a few milliseconds before the input's expected arrival.

Download Full-text

A Quaternionic Rate-Based Synaptic Learning Rule Derived from Spike-Timing Dependent Plasticity

Advances in Neural Networks - ISNN 2017 - Lecture Notes in Computer Science ◽

10.1007/978-3-319-59072-1_54 ◽

2017 ◽

pp. 457-465

Author(s):

Guang Qiao ◽

Hongyue Du ◽

Yi Zeng

Keyword(s):

Learning Rule ◽

Spike Timing ◽

Spike Timing Dependent Plasticity ◽

Dependent Plasticity ◽

Synaptic Learning

Download Full-text

Multiple topological representation self-organized by spike-timing-dependent synaptic learning rule

Cognitive Neurodynamics ◽

10.1007/s11571-008-9072-y ◽

2008 ◽

Vol 3 (1) ◽

pp. 33-38

Author(s):

Yutaka Sakai ◽

Koji Wada

Keyword(s):

Learning Rule ◽

Spike Timing ◽

Topological Representation ◽

Self Organized ◽

Synaptic Learning

Download Full-text

Spike-timing-dependent BDNF secretion and synaptic plasticity

Philosophical Transactions of the Royal Society B Biological Sciences ◽

10.1098/rstb.2013.0132 ◽

2014 ◽

Vol 369 (1633) ◽

pp. 20130132 ◽

Cited By ~ 27

Author(s):

Hui Lu ◽

Hyungju Park ◽

Mu-Ming Poo

Keyword(s):

Hippocampal Neurons ◽

Fluorescent Protein ◽

Time Window ◽

Hippocampal Slices ◽

Long Term Potentiation ◽

Spike Timing ◽

Dependent Manner ◽

Induced Responses ◽

Before And After ◽

Green Fluorescent

In acute hippocampal slices, we found that the presence of extracellular brain-derived neurotrophic factor (BDNF) is essential for the induction of spike-timing-dependent long-term potentiation (tLTP). To determine whether BDNF could be secreted from postsynaptic dendrites in a spike-timing-dependent manner, we used a reduced system of dissociated hippocampal neurons in culture. Repetitive pairing of iontophoretically applied glutamate pulses at the dendrite with neuronal spikes could induce persistent alterations of glutamate-induced responses at the same dendritic site in a manner that mimics spike-timing-dependent plasticity (STDP)—the glutamate-induced responses were potentiated and depressed when the glutamate pulses were applied 20 ms before and after neuronal spiking, respectively. By monitoring changes in the green fluorescent protein (GFP) fluorescence at the dendrite of hippocampal neurons expressing GFP-tagged BDNF, we found that pairing of iontophoretic glutamate pulses with neuronal spiking resulted in BDNF secretion from the dendrite at the iontophoretic site only when the glutamate pulses were applied within a time window of approximately 40 ms prior to neuronal spiking, consistent with the timing requirement of synaptic potentiation via STDP. Thus, BDNF is required for tLTP and BDNF secretion could be triggered in a spike-timing-dependent manner from the postsynaptic dendrite.

Download Full-text

Effects of memristive synapse radiation interactions on learning in spiking neural networks

SN Applied Sciences ◽

10.1007/s42452-021-04553-0 ◽

2021 ◽

Vol 3 (5) ◽

Author(s):

Sumedha Gandharava Dahl ◽

Robert C. Ivans ◽

Kurtis D. Cantley

Keyword(s):

Action Potentials ◽

Radiation Effects ◽

Synaptic Weight ◽

Learning Rule ◽

Spike Timing ◽

Spiking Neural Network ◽

Synaptic Action ◽

Drift Model ◽

Spatio Temporal ◽

Post Synaptic Neuron

AbstractThis study uses advanced modeling and simulation to explore the effects of external events such as radiation interactions on the synaptic devices in an electronic spiking neural network. Specifically, the networks are trained using the spike-timing-dependent plasticity (STDP) learning rule to recognize spatio-temporal patterns (STPs) representing 25 and 100-pixel characters. Memristive synapses based on a TiO2 non-linear drift model designed in Verilog-A are utilized, with STDP learning behavior achieved through bi-phasic pre- and post-synaptic action potentials. The models are modified to include experimentally observed state-altering and ionizing radiation effects on the device. It is found that radiation interactions tend to make the connection between afferents stronger by increasing the conductance of synapses overall, subsequently distorting the STDP learning curve. In the absence of consistent STPs, these effects accumulate over time and make the synaptic weight evolutions unstable. With STPs at lower flux intensities, the network can recover and relearn with constant training. However, higher flux can overwhelm the leaky integrate-and-fire post-synaptic neuron circuits and reduce stability of the network.

Download Full-text

R-Tempotron: A robust tempotron learning rule for spike timing-based decisions

2016 13th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) ◽

10.1109/iccwamtip.2016.8079823 ◽

2016 ◽

Cited By ~ 1

Author(s):

Malu Zhang ◽

Jianping Li ◽

Yangyang Wang ◽

Guoming Gao

Keyword(s):

Learning Rule ◽

Spike Timing

Download Full-text

A synaptic learning rule based on the temporal coincidence of pre- and postsynaptic activity

IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222) ◽

10.1109/ijcnn.2001.938981 ◽

2002 ◽

Cited By ~ 1

Author(s):

M.J. Denham ◽

S.L. Denham

Keyword(s):

Learning Rule ◽

Rule Based ◽

Postsynaptic Activity ◽

Synaptic Learning

Download Full-text

Biologically Plausible Sequence Learning with Spiking Neural Networks

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i02.5487 ◽

2020 ◽

Vol 34 (02) ◽

pp. 1316-1323

Author(s):

Zuozhu Liu ◽

Thiparat Chotibut ◽

Christopher Hillar ◽

Shaowei Lin

Keyword(s):

Neural Networks ◽

Sequence Learning ◽

Nervous Activity ◽

Learning Rule ◽

Generative Models ◽

Spike Timing ◽

Hopfield Network ◽

Spiking Neural Networks ◽

Theoretical Ground ◽

Time Model

Motivated by the celebrated discrete-time model of nervous activity outlined by McCulloch and Pitts in 1943, we propose a novel continuous-time model, the McCulloch-Pitts network (MPN), for sequence learning in spiking neural networks. Our model has a local learning rule, such that the synaptic weight updates depend only on the information directly accessible by the synapse. By exploiting asymmetry in the connections between binary neurons, we show that MPN can be trained to robustly memorize multiple spatiotemporal patterns of binary vectors, generalizing the ability of the symmetric Hopfield network to memorize static spatial patterns. In addition, we demonstrate that the model can efficiently learn sequences of binary pictures as well as generative models for experimental neural spike-train data. Our learning rule is consistent with spike-timing-dependent plasticity (STDP), thus providing a theoretical ground for the systematic design of biologically inspired networks with large and robust long-range sequence storage capacity.

Download Full-text