scholarly journals A Highly Robust Binary Neural Network Inference Accelerator Based on Binary Memristors

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2600
Author(s):  
Yiyang Zhao ◽  
Yongjia Wang ◽  
Ruibo Wang ◽  
Yuan Rong ◽  
Xianyang Jiang
Keyword(s):  
Neural Network ◽  
Network Inference ◽  
Circuit Complexity ◽  
Activation Function ◽  
Voltage Source ◽  
Data Set ◽  
Binary Neural Network ◽  
Application Potential ◽  
Differential Pair ◽  
Memristor Array

Since memristor was found, it has shown great application potential in neuromorphic computing. Currently, most neural networks based on memristors deploy the special analog characteristics of memristor. However, owing to the limitation of manufacturing process, non-ideal characteristics such as non-linearity, asymmetry, and inconsistent device periodicity appear frequently and definitely, therefore, it is a challenge to employ memristor in a massive way. On the contrary, a binary neural network (BNN) requires its weights to be either +1 or −1, which can be mapped by digital memristors with high technical maturity. Upon this, a highly robust BNN inference accelerator with binary sigmoid activation function is proposed. In the accelerator, the inputs of each network layer are either +1 or 0, which can facilitate feature encoding and reduce the peripheral circuit complexity of memristor hardware. The proposed two-column reference memristor structure together with current controlled voltage source (CCVS) circuit not only solves the problem of mapping positive and negative weights on memristor array, but also eliminates the sneak current effect under the minimum conductance status. Being compared to the traditional differential pair structure of BNN, the proposed two-column reference scheme can reduce both the number of memristors and the latency to refresh the memristor array by nearly 50%. The influence of non-ideal factors of memristor array such as memristor array yield, memristor conductance fluctuation, and reading noise on the accuracy of BNN is investigated in detail based on a newly memristor circuit model with non-ideal characteristics. The experimental results demonstrate that when the array yield α ≥ 5%, or the reading noise σ ≤ 0.25, a recognition accuracy greater than 97% on the MNIST data set is achieved.

Security of E-Health Systems Using Face Recognition Based on Convolutional Neural Network

2020 ◽  
Vol 2 (2) ◽  
pp. 37-41
Author(s):  
Zhixian Chen ◽  
Jialin Tang ◽  
Xueyuan Gong ◽  
Qinglang Su
Keyword(s):  
Neural Network ◽  
Face Recognition ◽  
Convolutional Neural Network ◽  
Principal Component ◽  
Activation Function ◽  
Support Vector ◽  
Data Set ◽  
Novel Approach ◽  
Rectified Linear Unit ◽  
The Face

In order to improve the low accuracy of the face recognition methods in the case of e-health, this paper proposed a novel face recognition approach, which is based on convolutional neural network (CNN). In detail, through resolving the convolutional kernel, rectified linear unit (ReLU) activation function, dropout, and batch normalization, this novel approach reduces the number of parameters of the CNN model, improves the non-linearity of the CNN model, and alleviates overfitting of the CNN model. In these ways, the accuracy of face recognition is increased. In the experiments, the proposed approach is compared with principal component analysis (PCA) and support vector machine (SVM) on ORL, Cohn-Kanade, and extended Yale-B face recognition data set, and it proves that this approach is promising.

scholarly journals Preference Neural Network

2021 ◽  
Author(s):  
Ayman Elgharabawy ◽  
Mukesh Prasad ◽  
Chin-Teng Lin
Keyword(s):  
Neural Network ◽  
Middle Layer ◽  
Activation Function ◽  
Ranking Problem ◽  
Data Set ◽  
Label Ranking ◽  
Speed Up ◽  
Single Data ◽  
Preference Orders ◽  
Multiple Value

Equality and incomparability multi-label ranking have not been introduced to learning before. This paper proposes new native ranker neural network to address the problem of multi-label ranking including incomparable preference orders using a new activation and error functions and new architecture. Preference Neural Network PNN solves the multi-label ranking problem, where labels may have indifference preference orders or subgroups which are equally ranked. PNN is a nondeep, multiple-value neuron, single middle layer and one or more output layers network. PNN uses a novel positive smooth staircase (PSS) or smooth staircase (SS) activation function and represents preference orders and Spearman ranking correlation as objective functions. It is introduced in two types, Type A is traditional NN architecture and Type B uses expanding architecture by introducing new type of hidden neuron has multiple activation function in middle layer and duplicated output layers to reinforce the ranking by increasing the number of weights. PNN accepts single data instance as inputs and output neurons represent the number of labels and output value represents the preference value. PNN is evaluated using a new preference mining data set that contains repeated label values which have not experimented on before. SS and PS speed-up the learning and PNN outperforms five previously proposed methods for strict label ranking in terms of accurate results with high computational efficiency.

scholarly journals Memristor Based Binary Convolutional Neural Network Architecture With Configurable Neurons

2021 ◽  
Vol 15 ◽  
Author(s):  
Lixing Huang ◽  
Jietao Diao ◽  
Hongshan Nie ◽  
Wei Wang ◽  
Zhiwei Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
High Precision ◽  
Network Architecture ◽  
Large Scale ◽  
Network Performance ◽  
Recognition Accuracy ◽  
Activation Function ◽  
Data Set ◽  
Neuron Activation

The memristor-based convolutional neural network (CNN) gives full play to the advantages of memristive devices, such as low power consumption, high integration density, and strong network recognition capability. Consequently, it is very suitable for building a wearable embedded application system and has broad application prospects in image classification, speech recognition, and other fields. However, limited by the manufacturing process of memristive devices, high-precision weight devices are currently difficult to be applied in large-scale. In the same time, high-precision neuron activation function also further increases the complexity of network hardware implementation. In response to this, this paper proposes a configurable full-binary convolutional neural network (CFB-CNN) architecture, whose inputs, weights, and neurons are all binary values. The neurons are proportionally configured to two modes for different non-ideal situations. The architecture performance is verified based on the MNIST data set, and the influence of device yield and resistance fluctuations under different neuron configurations on network performance is also analyzed. The results show that the recognition accuracy of the 2-layer network is about 98.2%. When the yield rate is about 64% and the hidden neuron mode is configured as −1 and +1, namely ±1 MD, the CFB-CNN architecture achieves about 91.28% recognition accuracy. Whereas the resistance variation is about 26% and the hidden neuron mode configuration is 0 and 1, namely 01 MD, the CFB-CNN architecture gains about 93.43% recognition accuracy. Furthermore, memristors have been demonstrated as one of the most promising devices in neuromorphic computing for its synaptic plasticity. Therefore, the CFB-CNN architecture based on memristor is SNN-compatible, which is verified using the number of pulses to encode pixel values in this paper.

scholarly journals Construction of a far-ultraviolet all-sky map from an incomplete survey: application of a deep learning algorithm

2021 ◽  
Vol 502 (3) ◽  
pp. 3200-3209
Author(s):  
Young-Soo Jo ◽  
Yeon-Ju Choi ◽  
Min-Gi Kim ◽  
Chang-Ho Woo ◽  
Kyoung-Wook Min ◽  
...  
Keyword(s):  
Neural Network ◽  
Galaxy Evolution ◽  
Network Architecture ◽  
Learning Algorithm ◽  
Activation Function ◽  
Dense Layer ◽  
Optical Parameters ◽  
Data Set ◽  
Deep Learning Algorithm ◽  
Far Ultraviolet

ABSTRACT We constructed a far-ultraviolet (FUV) all-sky map based on observations from the Far Ultraviolet Imaging Spectrograph (FIMS) aboard the Korean microsatellite Science and Technology SATellite-1. For the ${\sim}20{{\ \rm per\ cent}}$ of the sky not covered by FIMS observations, predictions from a deep artificial neural network were used. Seven data sets were chosen for input parameters, including five all-sky maps of H α, E(B − V), N(H i), and two X-ray bands, with Galactic longitudes and latitudes. 70 ${{\ \rm per\ cent}}$ of the pixels of the observed FIMS data set were randomly selected for training as target parameters and the remaining 30 ${{\ \rm per\ cent}}$ were used for validation. A simple four-layer neural network architecture, which consisted of three convolution layers and a dense layer at the end, was adopted, with an individual activation function for each convolution layer; each convolution layer was followed by a dropout layer. The predicted FUV intensities exhibited good agreement with Galaxy Evolution Explorer observations made in a similar FUV wavelength band for high Galactic latitudes. As a sample application of the constructed map, a dust scattering simulation was conducted with model optical parameters and a Galactic dust model for a region that included observed and predicted pixels. Overall, FUV intensities in the observed and predicted regions were reproduced well.

scholarly journals Deep Neural Network for Multi-Class Prediction of Student Performance in Educational Data

2019 ◽  
Vol 8 (2) ◽  
pp. 5073-5081
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Student Performance ◽  
Activation Function ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Fine Tuning ◽  
Academic Excellence ◽  
Data Set

Prediction of student performance is the significant part in processing the educational data. Machine learning algorithms are leading the role in this process. Deep learning is one of the important concepts of machine learning algorithm. In this paper, we applied the deep learning technique for prediction of the academic excellence of the students using R Programming. Keras and Tensorflow libraries utilized for making the model using neural network on the Kaggle dataset. The data is separated into testing data training data set. Plot the neural network model using neuralnet method and created the Deep Learning model using two hidden layers using ReLu activation function and one output layer using softmax activation function. After fine tuning process until the stable changes; this model produced accuracy as 85%.

scholarly journals Preference Neural Network

2020 ◽  
Author(s):  
Ayman Elgharabawy ◽  
Mukesh Parsad ◽  
Nikhil R. Pal ◽  
Chin-Teng Lin
Keyword(s):  
Neural Network ◽  
Middle Layer ◽  
Activation Function ◽  
Ranking Problem ◽  
Data Set ◽  
Label Ranking ◽  
Speed Up ◽  
Single Data ◽  
Preference Orders ◽  
Multiple Value

Equality and incomparability multi-label ranking have not been introduced to learning before. This paper proposes new native ranker neural network to address the problem of multi-label ranking including incomparable preference orders using a new activation and error functions and new architecture. Preference Neural Network PNN solves the multi-label ranking problem, where labels may have indifference preference orders or subgroups which are equally ranked. PNN is a nondeep, multiple-value neuron, single middle layer and one or more output layers network. PNN uses a novel positive smooth staircase (PSS) or smooth staircase (SS) activation function and represents preference orders and Spearman ranking correlation as objective functions. It is introduced in two types, Type A is traditional NN architecture and Type B uses expanding architecture by introducing new type of hidden neuron has multiple activation function in middle layer and duplicated output layers to reinforce the ranking by increasing the number of weights. PNN accepts single data instance as inputs and output neurons represent the number of labels and output value represents the preference value. PNN is evaluated using a new preference mining data set that contains repeated label values which have not experimented on before. SS and PS speed-up the learning and PNN outperforms five previously proposed methods for strict label ranking in terms of accurate results with high computational efficiency.

Sign in / Sign up

Export Citation Format

Share Document