Quantized Convolutional Neural Network Implementation on a Parallel-Connected Memristor Crossbar Array for Edge AI Platforms

2021 ◽  
Vol 21 (3) ◽  
pp. 1854-1861
Author(s):  
Jaeheum Lee ◽  
Jason K. Eshraghian ◽  
Sungjin Kim ◽  
Kamran Eshraghian ◽  
Kyoungrok Cho
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
High Precision ◽  
Synaptic Weight ◽  
High Resistance State ◽  
Artificial Neuron ◽  
Crossbar Array ◽  
Field Programmable ◽  
Resistance State ◽  
Memristor Crossbar

There are many challenges in the hardware implementation of a neural network using nanoscale memristor crossbar arrays where the use of analog cells is concerned. Multi-state or analog cells introduce more stringent noise margins, which are difficult to adhere to in light of variability. We propose a potential solution using a 1-bit memristor that stores binary values “0” or “1” with their memristive states, denoted as a high-resistance state (HRS) and a low-resistance state (LRS). In addition, we propose a new architecture consisting of 4-parallel 1-bit memristors at each crosspoint on the array. The four 1-bit memristors connected in parallel represent 5 decimal values according to the number of activated memristors. This is then mapped to a synaptic weight, which corresponds to the state of an artificial neuron in a neural network. We implement a convolutional neural network (CNN) model on a framework (tensorflow) using an equivalent quantized weight mapping model that demonstrates learning results almost identical to a high-precision CNN model. This radix-5 CNN is mapped to hardware on the proposed parallel-connected memristor crossbar array. Also, we propose a method for negative weight representation on a memristor crossbar array. Then, we verify the CNN hardware on an edge-AI (e-AI) platform, developed on a field-programmable gate array (FPGA). In this e-AI platform, we represent five weights per crosspoint using CLB logics. We test the learning results of the CNN hardware using an e-AI platform with a dataset consisting of 4×4 images in three classes. We verify the functionality of our radix-5 CNN implementation showing comparable classification accuracy to high-precision use cases, with reduction of the area of the memristor crossbar array by half, all verified on a FPGA. Implementing the CNN model on the FPGA board can contribute to the practical use of edge-AI.

scholarly journals Optimizing the Distribution of Memristance Values of Memristive Synapses for Reducing Power Consumption in Analog Memristor Crossbar-Based Neural Networks

2019 ◽  
Vol 9 (2) ◽  
pp. 2306-2309
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Power Consumption ◽  
Power Efficiency ◽  
Reducing Power ◽  
High Resistance State ◽  
Higher Power ◽  
Resistance State ◽  
Memristor Crossbar

Memristor circuits have become one of the potential hardware-based platforms for implementing artificial neural networks due to a lot of advantageous features. In this paper, we compare the power consumption between an analog memristor crossbar-based a binary memristor crossbar-based neural network for realizing a two-layer neural network and propose an efficient method for reducing the power consumption of the analog memristor crossbar-based neural network. A two-layer neural network is implemented using the memristor crossbar arrays, which can be used with analog synapse or binary synapse. For recognizing the test samples of MNIST dataset, the binary memristor crossbar-based neural work consumes higher power by 19% than the analog memristor-based neural network. The power consumption of the analog memristor crossbar-based neural network strongly depends on the distribution of memristance values and it can be reduced by optimizing the distribution of the memristance values. To improve the power efficiency, the bias resistance must be selected close to high resistance state. The power consumption of the analog memristor-based neural network is reduced by 86% when increasing the bias resistance from 20KΩ to 160KΩ. For the bias resistance of 160KΩ, analog memristor crossbar-based neural network consumes less power by 89% than the binary memristor crossbar-based neural network.

scholarly journals Artificial Neural Network for Response Inference of a Nonvolatile Resistance-Switch Array

Micromachines ◽  
2019 ◽  
Vol 10 (4) ◽  
pp. 219 ◽  
Author(s):  
Guhyun Kim ◽  
Vladimir Kornijcuk ◽  
Dohun Kim ◽  
Inho Kim ◽  
Cheol Hwang ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Applied Voltage ◽  
High Resistance State ◽  
Input Vector ◽  
Crossbar Array ◽  
Input And Output ◽  
Highly Nonlinear ◽  
Resistance State ◽  
Artificial Neural

An artificial neural network was utilized in the behavior inference of a random crossbar array (10 × 9 or 28 × 27 in size) of nonvolatile binary resistance-switches (in a high resistance state (HRS) or low resistance state (LRS)) in response to a randomly applied voltage array. The employed artificial neural network was a multilayer perceptron (MLP) with leaky rectified linear units. This MLP was trained with 500,000 or 1,000,000 examples. For each example, an input vector consisted of the distribution of resistance states (HRS or LRS) over a crossbar array plus an applied voltage array. That is, for a M × N array where voltages are applied to its M rows, the input vector was M × (N + 1) long. The calculated (correct) current array for each random crossbar array was used as data labels for supervised learning. This attempt was successful such that the correlation coefficient between inferred and correct currents reached 0.9995 for the larger crossbar array. This result highlights MLP that leverages its versatility to capture the quantitative linkage between input and output across the highly nonlinear crossbar array.

Wafer‐Scale 2D Hafnium Diselenide Based Memristor Crossbar Array for Energy‐Efficient Neural Network Hardware

2021 ◽  
pp. 2103376 ◽  
Author(s):  
Sifan Li ◽  
Mei‐Er Pam ◽  
Yesheng Li ◽  
Li Chen ◽  
Yu‐Chieh Chien ◽  
...  
Keyword(s):  
Neural Network ◽  
Energy Efficient ◽  
Wafer Scale ◽  
Crossbar Array ◽  
Neural Network Hardware ◽  
Memristor Crossbar

scholarly journals A High-Precision Magnetic-Assisted Heading Angle Calculation Method Based on a 1D Convolutional Neural Network (CNN) in a Complicated Magnetic Environment

Micromachines ◽  
2020 ◽  
Vol 11 (7) ◽  
pp. 642
Author(s):  
Guanghui Hu ◽  
Hong Wan ◽  
Xinxin Li
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
High Precision ◽  
Geomagnetic Field ◽  
Field Data ◽  
Time Window ◽  
Magnetic Data ◽  
Navigation Systems ◽  
Pedestrian Navigation ◽  
Heading Angle

Due to its widespread presence and independence from artificial signals, the application of geomagnetic field information in indoor pedestrian navigation systems has attracted extensive attention from researchers. However, for indoors environments, geomagnetic field signals can be severely disturbed by the complicated magnetic, leading to reduced positioning accuracy of magnetic-assisted navigation systems. Therefore, there is an urgent need for methods which screen out undisturbed geomagnetic field data for realizing the high accuracy pedestrian inertial navigation indoors. In this paper, we propose an algorithm based on a one-dimensional convolutional neural network (1D CNN) to screen magnetic field data. By encoding the magnetic data within a certain time window to a time series, a 1D CNN with two convolutional layers is designed to extract data features. In order to avoid errors arising from artificial labels, the feature vectors will be clustered in the feature space to classify the magnetic data using unsupervised methods. Our experimental results show that this method can distinguish the geomagnetic field data from indoors disturbed magnetic data well and further significantly improve the calculation accuracy of the heading angle. Our work provides a possible technical path for the realization of high-precision indoor pedestrian navigation systems.

Impact of resistive switching parameters on resistive random access memory crossbar arrays

2020 ◽  
Vol 34 (12) ◽  
pp. 2050115
Author(s):  
Liping Fu ◽  
Sikai Chen ◽  
Zewei Wu ◽  
Xiaoyan Li ◽  
Mingyang You ◽  
...  
Keyword(s):  
Resistive Switching ◽  
Random Access ◽  
High Resistance State ◽  
Resistive Random Access Memory ◽  
High Density ◽  
Access Memory ◽  
Storage Density ◽  
Crossbar Array ◽  
Resistance State ◽  
Potential Impact

Sneak current issue of RRAM-based crossbar array is one of the biggest hindrances for high-density memory application. The integration of an addition selector to each cell is one of the most familiar solutions to avoid this undesired cross-talk issue, and resistive switching parameters would affect on the storage density. This paper investigates the potential impact of different resistive switching parameters on crossbar arrays with one-diode one-resistor (1D1R) and one-selector one-resistor (1S1R) architectures. Results indicate that 1S1R architecture is a more scalable technology for high-density crossbar array than 1D1R, and the storage density of 1D1R- and 1S1R-based crossbar array shows little dependence on resistance values of high-resistance state and low-resistance state, which gives a guideline for choosing appropriate selectors for RRAM crossbar array with specific parameters.

A field-programmable gate array system for sonar image recognition based on convolutional neural network

2020 ◽  
pp. 095965182093934
Author(s):  
Chong Wang ◽  
Yu Jiang ◽  
Kai Wang ◽  
Fenglin Wei
Keyword(s):  
Neural Network ◽  
Energy Consumption ◽  
Convolutional Neural Network ◽  
Image Recognition ◽  
Field Programmable Gate Array ◽  
Oil And Gas ◽  
Sonar Image ◽  
Field Programmable ◽  
Sonar Images ◽  
Gate Array

Subsea pipeline is the safest, most reliable, and most economical way to transport oil and gas from an offshore platform to an onshore terminal. However, the pipelines may rupture under the harsh working environment, causing oil and gas leakage. This calls for a proper device and method to detect the state of subsea pipelines in a timely and precise manner. The autonomous underwater vehicle carrying side-scan sonar offers a desirable way for target detection in the complex environment under the sea. As a result, this article combines the field-programmable gate array, featuring high throughput, low energy consumption and a high degree of parallelism, and the convolutional neural network into a sonar image recognition system. First, a training set was constructed by screening and splitting the sonar images collected by sensors, and labeled one by one. Next, the convolutional neural network model was trained by the set on the workstation platform. The trained model was integrated into the field-programmable gate array system and applied to recognize actual datasets. The recognition results were compared with those of the workstation platform. The comparison shows that the computational precision of the designed field-programmable gate array system based on convolutional neural network is equivalent to that of the workstation platform; however, the recognition time of the designed system can be saved by more than 77%, and its energy consumption can also be saved by more than 96.67%. Therefore, our system basically satisfies our demand for energy-efficient, real-time, and accurate recognition of sonar images.

scholarly journals Memristor Based Binary Convolutional Neural Network Architecture With Configurable Neurons

2021 ◽  
Vol 15 ◽  
Author(s):  
Lixing Huang ◽  
Jietao Diao ◽  
Hongshan Nie ◽  
Wei Wang ◽  
Zhiwei Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
High Precision ◽  
Network Architecture ◽  
Large Scale ◽  
Network Performance ◽  
Recognition Accuracy ◽  
Activation Function ◽  
Data Set ◽  
Neuron Activation

The memristor-based convolutional neural network (CNN) gives full play to the advantages of memristive devices, such as low power consumption, high integration density, and strong network recognition capability. Consequently, it is very suitable for building a wearable embedded application system and has broad application prospects in image classification, speech recognition, and other fields. However, limited by the manufacturing process of memristive devices, high-precision weight devices are currently difficult to be applied in large-scale. In the same time, high-precision neuron activation function also further increases the complexity of network hardware implementation. In response to this, this paper proposes a configurable full-binary convolutional neural network (CFB-CNN) architecture, whose inputs, weights, and neurons are all binary values. The neurons are proportionally configured to two modes for different non-ideal situations. The architecture performance is verified based on the MNIST data set, and the influence of device yield and resistance fluctuations under different neuron configurations on network performance is also analyzed. The results show that the recognition accuracy of the 2-layer network is about 98.2%. When the yield rate is about 64% and the hidden neuron mode is configured as −1 and +1, namely ±1 MD, the CFB-CNN architecture achieves about 91.28% recognition accuracy. Whereas the resistance variation is about 26% and the hidden neuron mode configuration is 0 and 1, namely 01 MD, the CFB-CNN architecture gains about 93.43% recognition accuracy. Furthermore, memristors have been demonstrated as one of the most promising devices in neuromorphic computing for its synaptic plasticity. Therefore, the CFB-CNN architecture based on memristor is SNN-compatible, which is verified using the number of pulses to encode pixel values in this paper.

scholarly journals Simplified-Boost Reinforced Model-Based Complex Wind Signal Forecasting

2020 ◽  
Vol 2020 ◽  
pp. 1-16
Author(s):  
Qiushuang Lin ◽  
Chunxiang Li ◽  
Chao Wu
Keyword(s):  
Neural Network ◽  
Structural Health Monitoring ◽  
Convolutional Neural Network ◽  
Monitoring System ◽  
High Precision ◽  
Health Monitoring ◽  
Wind Engineering ◽  
Stage Model ◽  
Health Monitoring System ◽  
Non Gaussian

Wind signal forecasting has become more and more crucial in the structural health monitoring system and wind engineering recently. It is a challenging subject owing to the complicated volatility of wind signals. The robustness and generalization of a predictor are significant as well as of high precision. In this paper, an adaptive residual convolutional neural network (CNN) is developed, aiming at achieving not only high precision but also high adaptivity for various wind signals with varying complexity. Afterwards, reinforced forecasting is adopted to enhance the robustness of the preliminary forecasting. The preliminary forecast results by adaptive residual CNN are integrated with historical observed signals as the new input to reconstruct a new forecasting mapping. Meanwhile, simplified-boost strategy is applied for more generalized results. The results of multistep forecasting for five kinds of nonstationary non-Gaussian wind signals prove the more excellent adaptivity and robustness of the developed two-stage model compared with single models.

Sign in / Sign up

Export Citation Format

Share Document