scholarly journals A Brain-Inspired Homeostatic Neuron Based on Phase-Change Memories for Efficient Neuromorphic Computing

2021 ◽  
Vol 15 ◽  
Author(s):  
Irene Muñoz-Martin ◽  
Stefano Bianchi ◽  
Shahin Hashemkhani ◽  
Giacomo Pedretti ◽  
Octavian Melnic ◽  
...  
Keyword(s):  
Neural Network ◽  
Phase Change ◽  
Internal State ◽  
Dynamic Networks ◽  
Spike Timing ◽  
Learning Activity ◽  
Neuromorphic Computing ◽  
Learning Capability ◽  
Artificial Neuron ◽  
Improved Learning

One of the main goals of neuromorphic computing is the implementation and design of systems capable of dynamic evolution with respect to their own experience. In biology, synaptic scaling is the homeostatic mechanism which controls the frequency of neural spikes within stable boundaries for improved learning activity. To introduce such control mechanism in a hardware spiking neural network (SNN), we present here a novel artificial neuron based on phase change memory (PCM) devices capable of internal regulation via homeostatic and plastic phenomena. We experimentally show that this mechanism increases the robustness of the system thus optimizing the multi-pattern learning under spike-timing-dependent plasticity (STDP). It also improves the continual learning capability of hybrid supervised-unsupervised convolutional neural networks (CNNs), in terms of both resilience and accuracy. Furthermore, the use of neurons capable of self-regulating their fire responsivity as a function of the PCM internal state enables the design of dynamic networks. In this scenario, we propose to use the PCM-based neurons to design bio-inspired recurrent networks for autonomous decision making in navigation tasks. The agent relies on neuronal spike-frequency adaptation (SFA) to explore the environment via penalties and rewards. Finally, we show that the conductance drift of the PCM devices, contrarily to the applications in neural network accelerators, can improve the overall energy efficiency of neuromorphic computing by implementing bio-plausible active forgetting.

scholarly journals Graphene-based 3D XNOR-VRRAM with ternary precision for neuromorphic computing

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Batyrbek Alimkhanuly ◽  
Joon Sohn ◽  
Ik-Joon Chang ◽  
Seunghyun Lee
Keyword(s):  
Neural Network ◽  
Energy Consumption ◽  
Recognition Accuracy ◽  
Material Selection ◽  
Weighted Sum ◽  
Device Design ◽  
Key Factors ◽  
Neuromorphic Computing ◽  
Device Scaling ◽  
The Impact

AbstractRecent studies on neural network quantization have demonstrated a beneficial compromise between accuracy, computation rate, and architecture size. Implementing a 3D Vertical RRAM (VRRAM) array accompanied by device scaling may further improve such networks’ density and energy consumption. Individual device design, optimized interconnects, and careful material selection are key factors determining the overall computation performance. In this work, the impact of replacing conventional devices with microfabricated, graphene-based VRRAM is investigated for circuit and algorithmic levels. By exploiting a sub-nm thin 2D material, the VRRAM array demonstrates an improved read/write margins and read inaccuracy level for the weighted-sum procedure. Moreover, energy consumption is significantly reduced in array programming operations. Finally, an XNOR logic-inspired architecture designed to integrate 1-bit ternary precision synaptic weights into graphene-based VRRAM is introduced. Simulations on VRRAM with metal and graphene word-planes demonstrate 83.5 and 94.1% recognition accuracy, respectively, denoting the importance of material innovation in neuromorphic computing.

scholarly journals Biological modelling of a computational spiking neural network with neuronal avalanches

2017 ◽  
Vol 375 (2096) ◽  
pp. 20160286 ◽  
Author(s):  
Xiumin Li ◽  
Qing Chen ◽  
Fangzheng Xue
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Critical State ◽  
Spike Timing ◽  
Spiking Neural Networks ◽  
Mathematical Methods ◽  
Computational Accuracy ◽  
Active Neuron ◽  
Computational Performance ◽  
Neuronal Avalanches

In recent years, an increasing number of studies have demonstrated that networks in the brain can self-organize into a critical state where dynamics exhibit a mixture of ordered and disordered patterns. This critical branching phenomenon is termed neuronal avalanches. It has been hypothesized that the homeostatic level balanced between stability and plasticity of this critical state may be the optimal state for performing diverse neural computational tasks. However, the critical region for high performance is narrow and sensitive for spiking neural networks (SNNs). In this paper, we investigated the role of the critical state in neural computations based on liquid-state machines, a biologically plausible computational neural network model for real-time computing. The computational performance of an SNN when operating at the critical state and, in particular, with spike-timing-dependent plasticity for updating synaptic weights is investigated. The network is found to show the best computational performance when it is subjected to critical dynamic states. Moreover, the active-neuron-dominant structure refined from synaptic learning can remarkably enhance the robustness of the critical state and further improve computational accuracy. These results may have important implications in the modelling of spiking neural networks with optimal computational performance. This article is part of the themed issue ‘Mathematical methods in medicine: neuroscience, cardiology and pathology’.

scholarly journals Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Changming Wu ◽  
Heshan Yu ◽  
Seokhyeong Lee ◽  
Ruoming Peng ◽  
Ichiro Takeuchi ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Phase Change ◽  
Convolutional Neural Network ◽  
Large Scale ◽  
Phase Change Materials ◽  
Refractive Index Change ◽  
Optical Computing ◽  
Machine Learning Algorithms ◽  
Matrix Vector Multiplication

AbstractNeuromorphic photonics has recently emerged as a promising hardware accelerator, with significant potential speed and energy advantages over digital electronics for machine learning algorithms, such as neural networks of various types. Integrated photonic networks are particularly powerful in performing analog computing of matrix-vector multiplication (MVM) as they afford unparalleled speed and bandwidth density for data transmission. Incorporating nonvolatile phase-change materials in integrated photonic devices enables indispensable programming and in-memory computing capabilities for on-chip optical computing. Here, we demonstrate a multimode photonic computing core consisting of an array of programable mode converters based on on-waveguide metasurfaces made of phase-change materials. The programmable converters utilize the refractive index change of the phase-change material Ge2Sb2Te5 during phase transition to control the waveguide spatial modes with a very high precision of up to 64 levels in modal contrast. This contrast is used to represent the matrix elements, with 6-bit resolution and both positive and negative values, to perform MVM computation in neural network algorithms. We demonstrate a prototypical optical convolutional neural network that can perform image processing and recognition tasks with high accuracy. With a broad operation bandwidth and a compact device footprint, the demonstrated multimode photonic core is promising toward large-scale photonic neural networks with ultrahigh computation throughputs.

Dynamical Constraints on Using Precise Spike Timing to Compute in Recurrent Cortical Networks

2008 ◽  
Vol 20 (4) ◽  
pp. 974-993 ◽  
Author(s):  
Arunava Banerjee ◽  
Peggy Seriès ◽  
Alexandre Pouget
Keyword(s):  
Dynamical System ◽  
Initial Conditions ◽  
Internal State ◽  
Spike Timing ◽  
Spiking Neurons ◽  
Spatiotemporal Patterns ◽  
Cortical Networks ◽  
Temporal Precision ◽  
Cortical Areas ◽  
Dynamical Constraints

Several recent models have proposed the use of precise timing of spikes for cortical computation. Such models rely on growing experimental evidence that neurons in the thalamus as well as many primary sensory cortical areas respond to stimuli with remarkable temporal precision. Models of computation based on spike timing, where the output of the network is a function not only of the input but also of an independently initializable internal state of the network, must, however, satisfy a critical constraint: the dynamics of the network should not be sensitive to initial conditions. We have previously developed an abstract dynamical system for networks of spiking neurons that has allowed us to identify the criterion for the stationary dynamics of a network to be sensitive to initial conditions. Guided by this criterion, we analyzed the dynamics of several recurrent cortical architectures, including one from the orientation selectivity literature. Based on the results, we conclude that under conditions of sustained, Poisson-like, weakly correlated, low to moderate levels of internal activity as found in the cortex, it is unlikely that recurrent cortical networks can robustly generate precise spike trajectories, that is, spatiotemporal patterns of spikes precise to the millisecond timescale.

scholarly journals Emulation with Organic Memristive Devices of Impairment of LTP Mechanism in Neurodegenerative Disease Pathology

2017 ◽  
Vol 2017 ◽  
pp. 1-8 ◽  
Author(s):  
Silvia Battistoni ◽  
Victor Erokhin ◽  
Salvatore Iannotta
Keyword(s):  
Neural Network ◽  
Information Transmission ◽  
Common Sense ◽  
Artificial Neuron ◽  
Cell Network ◽  
Neuron Networks ◽  
A Cell ◽  
The Common ◽  
Major Progress ◽  
Single Synapse

We explore and demonstrate the extension of the synapse-mimicking properties of memristive devices to a dysfunctional synapse as it occurs in the Alzheimer’s disease (AD) pathology. The ability of memristive devices to reproduce synapse properties such as LTP, LTD, and STDP has been already widely demonstrated, and moreover, they were used for developing artificial neuron networks (perceptrons) able to simulate the information transmission in a cell network. However, a major progress would be to extend the common sense of neuromorphic device even to the case of dysfunction of natural synapses. Can memristors efficiently simulate them? We provide here evidences of the ability of emulating the dysfunctional synaptic behavior typical of the AD pathology with organic memristive devices considering the effect of the disease not only on a single synapse but also in the case of a neural network, composed by numerous synapses.

Reliable analog resistive switching behaviors achieved using memristive devices in AlOx/HfOx bilayer structure for neuromorphic systems

2021 ◽  
Author(s):  
Meng Qi ◽  
Tianquan Fu ◽  
Huadong Yang ◽  
ye tao ◽  
Chunran Li ◽  
...  
Keyword(s):  
Resistive Switching ◽  
Learning Experience ◽  
Random Access ◽  
Resistive Random Access Memory ◽  
Spike Timing ◽  
Information Storage ◽  
Bilayer Structure ◽  
Neuromorphic Computing ◽  
Von Neumann ◽  
Simulation Based

Abstract Human brain synaptic memory simulation based on resistive random access memory (RRAM) has an enormous potential to replace traditional Von Neumann digital computer thanks to several advantages, including its simple structure, high-density integration, and the capability to information storage and neuromorphic computing. Herein, the reliable resistive switching (RS) behaviors of RRAM are demonstrated by engineering AlOx/HfOx bilayer structure. This allows for uniform multibit information storage. Further, the analog switching behaviors are capable of imitate several synaptic learning functions, including learning experience behaviors, short-term plasticity-long-term plasticity transition, and spike-timing-dependent-plasticity (STDP). In addition, the memristor based on STDP learning rules are implemented in image pattern recognition. These results may offer a promising potential of HfOx-based memristors for future information storage and neuromorphic computing applications.

scholarly journals ПІДХОДИ ДО СТВОРЕННЯ ПРОГРАМНОГО АГЕНТА НА ОСНОВІ МЕТАЕВРИСТИК І ШТУЧНИХ НЕЙРОННИХ МЕРЕЖ

2019 ◽  
pp. 58-65
Author(s):  
Євген Євгенович Федоров ◽  
Марина Володимирівна Чичужко ◽  
Владислав Олегович Чичужко
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Internal State ◽  
Numerical Study ◽  
Agent Based ◽  
Development Framework ◽  
Number Of Visits ◽  
Artificial Neural ◽  
The Creation ◽  
Elman Recurrent Neural Network

In this article, has been developed a software agent based on meta-heuristics and artificial neural networks. The analysis of existing classes of agents and the selected reactive agent with internal state, which is well suited for partially observable, dynamic and non-episodic media, was carried out, and this agent has an internal state that preserves the state of the environment, obtained on the basis of the history of acts of perception, in the form of structured data. Were proposed approaches to create an agent based on meta-heuristics and an agent based on an artificial neural network. The development of reactive agents with internal state, based on the PSO (particle swarm optimization) metaheuristics, which are related to individual particles and to a whole swarm and interact by messages was proposed. Also, has been proposed an approach to the creation of a reactive agent with an internal state based on the Elman recurrent neural network. The agent-based approach allows combining different areas of artificial intelligence, digital signal processing, mathematical modeling, and game theory. The proposed agents were implemented using the JADE (Java Agent Development Framework) toolkit, which is one of the most popular tools for the creation of agent systems. A numerical study was made to determine the parameters of the swarm PSO metaheuristics and the Elman recurrent neural network. As a purpose function, the Rastrigin test function has been used. The number of visits to the website of DonNTU was used as an input sample for the Elman network. The minimum average square error forecast was the criterion for choosing the structure of a network model. 10 hiding neurons were used to predict the number of visits to the website page, since, with increasing of hidden neurons number, the change in the error value is small. To determine the number of particles in the swarm, a series of experiments was conducted, the results of which are presented by graphs. The proposed approaches can be used in intelligent computer systems.

Sign in / Sign up

Export Citation Format

Share Document