Dynamical Characteristics of Recurrent Neuronal Networks Are Robust Against Low Synaptic Weight Resolution

The representation of the natural-density, heterogeneous connectivity of neuronal network models at relevant spatial scales remains a challenge for Computational Neuroscience and Neuromorphic Computing. In particular, the memory demands imposed by the vast number of synapses in brain-scale network simulations constitute a major obstacle. Limiting the number resolution of synaptic weights appears to be a natural strategy to reduce memory and compute load. In this study, we investigate the effects of a limited synaptic-weight resolution on the dynamics of recurrent spiking neuronal networks resembling local cortical circuits and develop strategies for minimizing deviations from the dynamics of networks with high-resolution synaptic weights. We mimic the effect of a limited synaptic weight resolution by replacing normally distributed synaptic weights with weights drawn from a discrete distribution, and compare the resulting statistics characterizing firing rates, spike-train irregularity, and correlation coefficients with the reference solution. We show that a naive discretization of synaptic weights generally leads to a distortion of the spike-train statistics. If the weights are discretized such that the mean and the variance of the total synaptic input currents are preserved, the firing statistics remain unaffected for the types of networks considered in this study. For networks with sufficiently heterogeneous in-degrees, the firing statistics can be preserved even if all synaptic weights are replaced by the mean of the weight distribution. We conclude that even for simple networks with non-plastic neurons and synapses, a discretization of synaptic weights can lead to substantial deviations in the firing statistics unless the discretization is performed with care and guided by a rigorous validation process. For the network model used in this study, the synaptic weights can be replaced by low-resolution weights without affecting its macroscopic dynamical characteristics, thereby saving substantial amounts of memory.

Crosstalk between biochemical signaling and trafficking determines AMPAR dynamics in synaptic plasticity

Synaptic plasticity involves the modification of both biochemical and structural components of neurons. Many studies have revealed that the change in the number density of the glutamatergic receptor AMPAR at the synapse is proportional to synaptic weight update; increase in AMPAR corresponds to strengthening of synapses while decrease in AMPAR density weakens synaptic connections. The dynamics of AMPAR are thought to be regulated by upstream signaling, primarily the calcium-CaMKII pathway, trafficking to and from the synapse, and influx from extrasynaptic sources. Here, we have developed a set of models using compartmental ordinary differential equations to systematically investigate contributions of signaling and trafficking variations on AMPAR dynamics at the synaptic site. We find that the model properties including network architecture and parameters significantly affect the integration of fast upstream species by slower downstream species. Furthermore, we predict that the model outcome, as determined by bound AMPAR at the synaptic site, depends on (a) the choice of signaling model (bistable CaMKII or monostable CaMKII dynamics), (b) trafficking versus influx contributions, and (c) frequency of stimulus. Therefore, AMPAR dynamics can have unexpected dependencies when upstream signaling dynamics (such as CaMKII and PP1) are coupled with trafficking modalities.

An Analysis on the Architecture and the Size of Quantized Hardware Neural Networks Based on Memristors

We have performed different simulation experiments in relation to hardware neural networks (NN) to analyze the role of the number of synapses for different NN architectures in the network accuracy, considering different datasets. A technology that stands upon 4-kbit 1T1R ReRAM arrays, where resistive switching devices based on HfO2 dielectrics are employed, is taken as a reference. In our study, fully dense (FdNN) and convolutional neural networks (CNN) were considered, where the NN size in terms of the number of synapses and of hidden layer neurons were varied. CNNs work better when the number of synapses to be used is limited. If quantized synaptic weights are included, we observed that NN accuracy decreases significantly as the number of synapses is reduced; in this respect, a trade-off between the number of synapses and the NN accuracy has to be achieved. Consequently, the CNN architecture must be carefully designed; in particular, it was noticed that different datasets need specific architectures according to their complexity to achieve good results. It was shown that due to the number of variables that can be changed in the optimization of a NN hardware implementation, a specific solution has to be worked in each case in terms of synaptic weight levels, NN architecture, etc.

Bidirectional synaptic plasticity rapidly modifies hippocampal representations

Learning requires neural adaptations thought to be mediated by activity-dependent synaptic plasticity. A relatively non-standard form of synaptic plasticity driven by dendritic calcium spikes, or plateau potentials, has been reported to underlie place field formation in rodent hippocampal CA1 neurons. Here we found that this behavioral timescale synaptic plasticity (BTSP) can also reshape existing place fields via bidirectional synaptic weight changes that depend on the temporal proximity of plateau potentials to pre-existing place fields. When evoked near an existing place field, plateau potentials induced less synaptic potentiation and more depression, suggesting BTSP might depend inversely on postsynaptic activation. However, manipulations of place cell membrane potential and computational modeling indicated that this anti-correlation actually results from a dependence on current synaptic weight such that weak inputs potentiate and strong inputs depress. A network model implementing this bidirectional synaptic learning rule suggested that BTSP enables population activity, rather than pairwise neuronal correlations, to drive neural adaptations to experience.

Hamiltonian energy and coexistence of hidden firing patterns from bidirectional coupling between two different neurons

In this paper, bidirectional-coupled neurons through an asymmetric electrical synapse are investigated. These coupled neurons involve 2D Hindmarsh–Rose (HR) and 2D FitzHugh–Nagumo (FN) neurons. The equilibria of the coupled neurons model are investigated, and their stabilities have revealed that, for some values of the electrical synaptic weight, the model under consideration can display either self-excited or hidden firing patterns. In addition, the hidden coexistence of chaotic bursting with periodic spiking, chaotic spiking with period spiking, chaotic bursting with a resting pattern, and the coexistence of chaotic spiking with a resting pattern are also found for some sets of electrical synaptic coupling. For all the investigated phenomena, the Hamiltonian energy of the model is computed. It enables the estimation of the amount of energy released during the transition between the various electrical activities. Pspice simulations are carried out based on the analog circuit of the coupled neurons to support our numerical results. Finally, an STM32F407ZE microcontroller development board is exploited for the digital implementation of the proposed coupled neurons model.

Alternative negative weight for simpler hardware implementation of synapse device based neuromorphic system

Lately, there has been a rapid increase in the use of software-based deep learning neural networks (S-DNN) for the analysis of unstructured data consumption. For implementation of the S-DNN, synapse-device-based hardware DNN (H-DNN) has been proposed as an alternative to typical Von-Neumann structural computing systems. In the H-DNN, various numerical values such as the synaptic weight, activation function, and etc., have to be realized through electrical device or circuit. Among them, the synaptic weight that should have both positive and negative numerical values needs to be implemented in a simpler way. Because the synaptic weight has been expressed by conductance value of the synapse device, it always has a positive value. Therefore, typically, a pair of synapse devices is required to realize the negative weight values, which leads to additional hardware resources such as more devices, higher power consumption, larger area, and increased circuit complexity. Herein, we propose an alternative simpler method to realize the negative weight (named weight shifter) and its hardware implementation. To demonstrate the weight shifter, we investigated its theoretical, numerical, and circuit-related aspects, following which the H-DNN circuit was successfully implemented on a printed circuit board.

Design of Synaptic Neuronal Circuit Based on Memristors

To solve the problems of poor learning efficiency and low accuracy caused by the single fixed synaptic weight in the traditional artificial neural network. On the foundation of the improved memristor model, this paper designs a synaptic neuronal circuit based on the natural memory characteristics of the memristor. This synapse is composed of six memristors. The resistance of the memristor is changed by adding a periodic square wave to update the synaptic weight. This circuit can realize signed synaptic weighting, which has certain linear characteristics. Finally, two synaptic weight update methods are proposed based on this circuit, and the validity of the design is verified through Spice simulation experiments.

A Biologically Plausible Model for Continual Learning using Synaptic Weight Attractors

The human brain readily learns tasks in sequence without forgetting previous ones. Artificial neural networks (ANNs), on the other hand, need to be modified to achieve similar performance. While effective, many algorithms that accomplish this are based on weight importance methods that do not correspond to biological mechanisms. Here we introduce a simple, biologically plausible method for enabling effective continual learning in ANNs. We show that it is possible to learn a weight-dependent plasticity function that prevents catastrophic forgetting over multiple tasks. We highlight the effectiveness of our method by evaluating it on a set of MNIST classification tasks. We further find that the use of our method promotes synaptic multi-modality, similar to that seen in biology.

Coexisting Infinitely Many Nonchaotic Attractors in a Memristive Weight-Based Tabu Learning Neuron

Memristive synaptic weight is a changeable connection synaptic weight. It reflects the self-adaption physical processing in biological neurons. To study its dynamical effect, this paper presents a memristive synaptic weight-based tabu learning neuron model. It is constructed by replacing the resistive self-connection synaptic weight in the tabu learning neuron with a memristive self-connection synaptic weight. The equilibrium point of the memristive tabu learning model is time-varying and switches between no equilibrium state and line equilibrium state with the change of the external current. Particularly, the stability of the line equilibrium state closely relies on the initial state of the memristor, resulting in the emergence of coexisting infinitely many nonchaotic attractors. By employing the bifurcation plots, Lyapunov exponents, and phase plots, this paper numerically reveals the initial state-switched coexisting bifurcation behaviors and initial state-relied extreme multistability, and thereby discloses the coexisting infinitely many nonchaotic attractors composed of mono-periodic, multiperiodic, and quasi-periodic orbits. In addition, PSIM circuit simulations and printed-circuit board-based experiments are executed and the coexisting infinitely many nonchaotic attractors are realized physically. The results well verify the numerical simulations.