scholarly journals Spiking Neural Network (SNN) With Memristor Synapses Having Non-linear Weight Update

2021 ◽  
Vol 15 ◽  
Author(s):  
Taeyoon Kim ◽  
Suman Hu ◽  
Jaewook Kim ◽  
Joon Young Kwak ◽  
Jongkil Park ◽  
...  
Keyword(s):  
Neural Network ◽  
Energy Efficient ◽  
Hardware Implementation ◽  
Promising Candidate ◽  
Spike Neural Network ◽  
Neuromorphic Hardware ◽  
Non Linear ◽  
Device Properties ◽  
The Impact ◽  
The Brain

Among many artificial neural networks, the research on Spike Neural Network (SNN), which mimics the energy-efficient signal system in the brain, is drawing much attention. Memristor is a promising candidate as a synaptic component for hardware implementation of SNN, but several non-ideal device properties are making it challengeable. In this work, we conducted an SNN simulation by adding a device model with a non-linear weight update to test the impact on SNN performance. We found that SNN has a strong tolerance for the device non-linearity and the network can keep the accuracy high if a device meets one of the two conditions: 1. symmetric LTP and LTD curves and 2. positive non-linearity factors for both LTP and LTD. The reason was analyzed in terms of the balance between network parameters as well as the variability of weight. The results are considered to be a piece of useful prior information for the future implementation of emerging device-based neuromorphic hardware.

Chapter 9. Spike-Based Symbolic Computations on Bit Strings and Numbers

2021 ◽  
Author(s):  
Ceca Kraišniković ◽  
Wolfgang Maass ◽  
Robert Legenstein
Keyword(s):  
Neural Networks ◽  
Energy Efficient ◽  
Learning Rule ◽  
Spiking Neural Networks ◽  
New Paradigm ◽  
Symbolic Computations ◽  
Neuromorphic Hardware ◽  
State Of Research ◽  
Higher Cognitive Functions ◽  
The Brain

The brain uses recurrent spiking neural networks for higher cognitive functions such as symbolic computations, in particular, mathematical computations. We review the current state of research on spike-based symbolic computations of this type. In addition, we present new results which show that surprisingly small spiking neural networks can perform symbolic computations on bit sequences and numbers and even learn such computations using a biologically plausible learning rule. The resulting networks operate in a rather low firing rate regime, where they could not simply emulate artificial neural networks by encoding continuous values through firing rates. Thus, we propose here a new paradigm for symbolic computation in neural networks that provides concrete hypotheses about the organization of symbolic computations in the brain. The employed spike-based network models are the basis for drastically more energy-efficient computer hardware – neuromorphic hardware. Hence, our results can be seen as creating a bridge from symbolic artificial intelligence to energy-efficient implementation in spike-based neuromorphic hardware.

Hardware Implementation of Energy Efficient Deep Learning Neural Network Based on Nanoscale Flash Computing Array

2019 ◽  
Vol 4 (5) ◽  
pp. 1800720 ◽  
Author(s):  
Yachen Xiang ◽  
Peng Huang ◽  
Runze Han ◽  
Zheng Zhou ◽  
Qingming Shu ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Energy Efficient ◽  
Hardware Implementation ◽  
Deep Learning Neural Network

Neuromorphic Memristive Chips: Design and Technology

2021 ◽  
Vol 23 (6) ◽  
pp. 285-294
Author(s):  
N.V. Andreeva ◽  
◽  
V.V. Luchinin ◽  
E.A. Ryndin ◽  
M.G. Anchkov ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Resistive Switching ◽  
Hardware Implementation ◽  
Functional Materials ◽  
Design And Technology ◽  
Spiking Neural Network ◽  
Chip Design ◽  
Neuromorphic Hardware ◽  
Metal Insulator Metal

Memristive neuromorphic chips exploit a prospective class of novel functional materials (memristors) to deploy a new architecture of spiking neural networks for developing basic blocks of brain-like systems. Memristor-based neuromorphic hardware solutions for multi-agent systems are considered as challenges in frontier areas of chip design for fast and energy-efficient computing. As functional materials, metal oxide thin films with resistive switching and memory effects (memristive structures) are recognized as a potential elemental base for new components of neuromorphic engineering, enabling a combination of both data storage and processing in a single unit. A key design issue in this case is a hardware defined functionality of neural networks. The gradient change of resistive properties of memristive elements and its non-volatile memory behavior ensure the possibility of spiking neural network organization with unsupervised learning through hardware implementation of basic synaptic mechanisms, such as Hebb's learning rules including spike — timing dependent plasticity, long-term potentiation and depression. This paper provides an overview of scientific researches carrying out at Saint Petersburg Electrotechnical University "LETI" since 2014 in the field of novel electronic components for neuromorphic hardware solutions of brain-like chip design. Among the most promising concepts developed by ETU "LETI" are: the design of metal-insulator-metal structures exhibiting multilevel resistive switching (gradient tuning of resistive properties and bipolar resistive switching are combined together in a sin¬gle memristive element) for further use as artificial synaptic devices in neuromorphic chips; computing schemes for spatio-temporal pattern recognition based on spiking neural network architecture implementation; breadboard models of analogue circuits of hardware implementation of neuromorphic blocks for brain-like system developing.

scholarly journals Spike-based symbolic computations on bit strings and numbers

2021 ◽  
Author(s):  
Ceca Kraisnikovic ◽  
Wolfgang Maass ◽  
Robert Legenstein
Keyword(s):  
Neural Networks ◽  
Energy Efficient ◽  
Learning Rule ◽  
Network Models ◽  
Spiking Neural Networks ◽  
Computer Hardware ◽  
New Paradigm ◽  
Symbolic Computations ◽  
Neuromorphic Hardware ◽  
The Brain

The brain uses recurrent spiking neural networks for higher cognitive functions such as symbolic computations, in particular, mathematical computations. We review the current state of research on spike-based symbolic computations of this type. In addition, we present new results which show that surprisingly small spiking neural networks can perform symbolic computations on bit sequences and numbers and even learn such computations using a biologically plausible learning rule. The resulting networks operate in a rather low firing rate regime, where they could not simply emulate artificial neural networks by encoding continuous values through firing rates. Thus, we propose here a new paradigm for symbolic computation in neural networks that provides concrete hypotheses about the organization of symbolic computations in the brain. The employed spike-based network models are the basis for drastically more energy-efficient computer hardware -- neuromorphic hardware. Hence, our results can be seen as creating a bridge from symbolic artificial intelligence to energy-efficient implementation in spike-based neuromorphic hardware.

ПРОГРАММНЫЕ И АППАРАТНЫЕ РЕАЛИЗАЦИИ ОСНОВНЫХ КОМПОНЕНТ СПАЙКОВЫХ И ФОРМАЛЬНЫХ НЕЙРОННЫХ СЕТЕЙ

2020 ◽  
Vol 96 (3s) ◽  
pp. 570-579
Author(s):  
И.А. Суражевский ◽  
К.Э. Никируй ◽  
А.В. Емельянов ◽  
В.В. Рыльков ◽  
В.А. Демин
Keyword(s):  
Neural Network ◽  
Numerical Simulation ◽  
Hardware Implementation ◽  
Hebbian Learning ◽  
Inhibitory Neurons ◽  
Local Weight ◽  
Network Algorithms ◽  
Spike Neural Network

Разработаны Verilog-A-модели тормозного и возбуждающего нейронов с бипрямоугольной и битреугольной формами импульсов (спайков) и мемристивными синаптическими весами. Показана возможность сходимости весов в численном моделировании обучения по Хеббу нейрона на основе локальных правил модификации весов. Предложена схема нейросинаптического ядра для аппаратной реализации формальных и спайковых нейронных сетей на их основе. The paper presents Verilog-A models of excitatory and inhibitory neurons with birectangular and bitriangular shapes of spikes. Besides, it highlights the possibility of convergence of weights in the numerical simulation of a neuron Hebbian learning based on local weight updates rules, as well as offers schemes for the neurosynaptic core for the hardware implementation of formal and spike neural network algorithms.

scholarly journals Simulation of the Spiking Neural Network based on Practical Memristor

2018 ◽  
Vol 173 ◽  
pp. 01025
Author(s):  
Xi Zhu ◽  
Yi Sun ◽  
Haijun Liu ◽  
Qingjiang Li ◽  
Hui Xu
Keyword(s):  
Neural Network ◽  
Recognition Task ◽  
Single Layer ◽  
Spiking Neural Network ◽  
Biological Structure ◽  
Biologically Inspired ◽  
The Impact ◽  
The Brain ◽  
Pattern Recognition Task ◽  
Significant Attention

In order to gain a better understanding of the brain and explore biologically-inspired computation, significant attention is being paid to research into the spike-based neural computation. Spiking neural network (SNN), which is inspired by the understanding of observed biological structure, has been increasingly applied to pattern recognition task. In this work, a single layer SNN architecture based on the characteristics of spiking timing dependent plasticity (STDP) in accordance with the actual test of the device data has been proposed. The device data is derived from the Ag/GeSe/TiN fabricated memristor. The network has been tested on the MNIST dataset, and the classification accuracy attains 90.2%. Furthermore, the impact of device instability on the SNN performance has been discussed, which can propose guidelines for fabricating memristors used for SNN architecture based on STDP characteristics.

scholarly journals Neurolinguistics: Structure, Function, and Connectivity in the Bilingual Brain

2016 ◽  
Vol 2016 ◽  
pp. 1-22 ◽  
Author(s):  
Becky Wong ◽  
Bin Yin ◽  
Beth O’Brien
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Developmental Psychology ◽  
Theoretical Models ◽  
Future Research ◽  
Universal Language ◽  
And Function ◽  
The Impact ◽  
The Brain ◽  
Neuroimaging Techniques

Advances in neuroimaging techniques and analytic methods have led to a proliferation of studies investigating the impact of bilingualism on the cognitive and brain systems in humans. Lately, these findings have attracted much interest and debate in the field, leading to a number of recent commentaries and reviews. Here, we contribute to the ongoing discussion by compiling and interpreting the plethora of findings that relate to the structural, functional, and connective changes in the brain that ensue from bilingualism. In doing so, we integrate theoretical models and empirical findings from linguistics, cognitive/developmental psychology, and neuroscience to examine the following issues: (1) whether the language neural network is different for first (dominant) versus second (nondominant) language processing; (2) the effects of bilinguals’ executive functioning on the structure and function of the “universal” language neural network; (3) the differential effects of bilingualism on phonological, lexical-semantic, and syntactic aspects of language processing on the brain; and (4) the effects of age of acquisition and proficiency of the user’s second language in the bilingual brain, and how these have implications for future research in neurolinguistics.

scholarly journals Learning to Approximate Functions Using Nb-Doped SrTiO3 Memristors

2021 ◽  
Vol 14 ◽  
Author(s):  
Thomas F. Tiotto ◽  
Anouk S. Goossens ◽  
Jelmer P. Borst ◽  
Tamalika Banerjee ◽  
Niels A. Taatgen
Keyword(s):  
Neural Network ◽  
Learning Algorithm ◽  
Universal Function ◽  
Learning Performance ◽  
Initial State ◽  
Neuromorphic Hardware ◽  
The Difference ◽  
Computing Platforms ◽  
Function Approximator ◽  
The Impact

Memristors have attracted interest as neuromorphic computation elements because they show promise in enabling efficient hardware implementations of artificial neurons and synapses. We performed measurements on interface-type memristors to validate their use in neuromorphic hardware. Specifically, we utilized Nb-doped SrTiO3 memristors as synapses in a simulated neural network by arranging them into differential synaptic pairs, with the weight of the connection given by the difference in normalized conductance values between the two paired memristors. This network learned to represent functions through a training process based on a novel supervised learning algorithm, during which discrete voltage pulses were applied to one of the two memristors in each pair. To simulate the fact that both the initial state of the physical memristive devices and the impact of each voltage pulse are unknown we injected noise into the simulation. Nevertheless, discrete updates based on local knowledge were shown to result in robust learning performance. Using this class of memristive devices as the synaptic weight element in a spiking neural network yields, to our knowledge, one of the first models of this kind, capable of learning to be a universal function approximator, and strongly suggests the suitability of these memristors for usage in future computing platforms.

Research on behavior recognition based on feature fusion of automatic coder and recurrent neural network

2020 ◽  
Vol 39 (6) ◽  
pp. 8927-8935
Author(s):  
Bing Zheng ◽  
Dawei Yun ◽  
Yan Liang
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Behavior Pattern ◽  
Rapid Development ◽  
Video Data ◽  
Support Vector ◽  
Behavior Recognition ◽  
Learning Methods ◽  
The Impact ◽  
Internet Of Things Technology

Under the impact of COVID-19, research on behavior recognition are highly needed. In this paper, we combine the algorithm of self-adaptive coder and recurrent neural network to realize the research of behavior pattern recognition. At present, most of the research of human behavior recognition is focused on the video data, which is based on the video number. At the same time, due to the complexity of video image data, it is easy to violate personal privacy. With the rapid development of Internet of things technology, it has attracted the attention of a large number of experts and scholars. Researchers have tried to use many machine learning methods, such as random forest, support vector machine and other shallow learning methods, which perform well in the laboratory environment, but there is still a long way to go from practical application. In this paper, a recursive neural network algorithm based on long and short term memory (LSTM) is proposed to realize the recognition of behavior patterns, so as to improve the accuracy of human activity behavior recognition.

Sign in / Sign up

Export Citation Format

Share Document