An Artificial Synaptic Plasticity Mechanism for Classical Conditioning with Neural Networks

AbstractThe recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the “Synaptic Plasticity” paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an “elite” ONN can then be configured using the top-ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result, the performance gap over the CNNs further widens.

Download Full-text

Neuromorphic Functional Modules of Spiking Neural Network

Nano- i Mikrosistemnaya Tehnika ◽

10.17587/nmst.23.317-326 ◽

2021 ◽

Vol 23 (6) ◽

pp. 317-326

Author(s):

E.A. Ryndin ◽

◽

N.V. Andreeva ◽

V.V. Luchinin ◽

K.S. Goncharov ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Artificial Neural Networks ◽

Synaptic Plasticity ◽

Data Processing ◽

Circuit Design ◽

Spiking Neural Networks ◽

Spiking Neural Network ◽

Wide Range ◽

Artificial Neural

In the current era, design and development of artificial neural networks exploiting the architecture of the human brain have evolved rapidly. Artificial neural networks effectively solve a wide range of common for artificial intelligence tasks involving data classification and recognition, prediction, forecasting and adaptive control of object behavior. Biologically inspired underlying principles of ANN operation have certain advantages over the conventional von Neumann architecture including unsupervised learning, architectural flexibility and adaptability to environmental change and high performance under significantly reduced power consumption due to heavy parallel and asynchronous data processing. In this paper, we present the circuit design of main functional blocks (neurons and synapses) intended for hardware implementation of a perceptron-based feedforward spiking neural network. As the third generation of artificial neural networks, spiking neural networks perform data processing utilizing spikes, which are discrete events (or functions) that take place at points in time. Neurons in spiking neural networks initiate precisely timing spikes and communicate with each other via spikes transmitted through synaptic connections or synapses with adaptable scalable weight. One of the prospective approach to emulate the synaptic behavior in hardware implemented spiking neural networks is to use non-volatile memory devices with analog conduction modulation (or memristive structures). Here we propose a circuit design for functional analogues of memristive structure to mimic a synaptic plasticity, pre- and postsynaptic neurons which could be used for developing circuit design of spiking neural network architectures with different training algorithms including spike-timing dependent plasticity learning rule. Two different circuits of electronic synapse were developed. The first one is an analog synapse with photoresistive optocoupler used to ensure the tunable conductivity for synaptic plasticity emulation. While the second one is a digital synapse, in which the synaptic weight is stored in a digital code with its direct conversion into conductivity (without digital-to-analog converter andphotoresistive optocoupler). The results of the prototyping of developed circuits for electronic analogues of synapses, pre- and postsynaptic neurons and the study of transient processes are presented. The developed approach could provide a basis for ASIC design of spiking neural networks based on CMOS (complementary metal oxide semiconductor) design technology.

Download Full-text

Plasticity: Cells, Circuits, Brains, and Behavior

The Computational Brain ◽

10.7551/mitpress/9780262533393.003.0005 ◽

2016 ◽

Author(s):

Patricia S. Churchland ◽

Terrence J. Sejnowski

Keyword(s):

Nervous System ◽

Synaptic Plasticity ◽

Classical Conditioning ◽

Neuronal Plasticity ◽

Neural Mechanism ◽

Synaptic Strength ◽

Physical Mechanisms ◽

And Behavior ◽

The Brain

This chapter examines the physical mechanisms in nervous systems in order to elucidate the structural bases and functional principles of synaptic plasticity. Neuroscientific research on plasticity can be divided into four main streams: the neural mechanism for relatively simple kinds of plasticity, such as classical conditioning or habituation; anatomical and physiological studies of temporal lobe structures, including the hippocampus and the amygdala; study of the development of the visual system; and the relation between the animal's genes and the development of its nervous system. The chapter first considers the role of the mammalian hippocampus in learning and memory before discussing Donald Hebb's views on synaptic plasticity. It then explores the mechanisms underlying neuronal plasticity and those that decrease synaptic strength, the relevance of time with respect to plasticity, and the occurrence of plasticity during the development of the nervous system. It also describes modules, modularity, and networks in the brain.

Download Full-text