scholarly journals An Approach of Binary Neural Network Energy-Efficient Implementation

Electronics ◽  
2021 ◽  
Vol 10 (15) ◽  
pp. 1830
Author(s):  
Jiabao Gao ◽  
Qingliang Liu ◽  
Jinmei Lai
Keyword(s):  
Neural Network ◽  
Energy Efficiency ◽  
Energy Efficient ◽  
State Of The Art ◽  
Smart Sensors ◽  
Real Output ◽  
Internal Memory ◽  
Binary Neural Network ◽  
Pruning Strategy ◽  
Better Than

Binarized neural networks (BNNs), which have 1-bit weights and activations, are well suited for FPGA accelerators as their dominant computations are bitwise arithmetic, and the reduction in memory requirements means that all the network parameters can be stored in internal memory. However, the energy efficiency of these accelerators is still restricted by the abundant redundancies in BNNs. This hinders their deployment for applications in smart sensors and tiny devices because these scenarios have tight constraints with respect to energy consumption. To overcome this problem, we propose an approach to implement BNN inference while offering excellent energy efficiency for the accelerators by means of pruning the massive redundant operations while maintaining the original accuracy of the networks. Firstly, inspired by the observation that the convolution processes of two related kernels contain many repeated computations, we first build one formula to clarify the reusing relationships between their convolutional outputs and remove the unnecessary operations. Furthermore, by generalizing this reusing relationship to one tile of kernels in one neuron, we adopt an inclusion pruning strategy to further skip the superfluous evaluations of the neurons whose real output values can be determined early. Finally, we evaluate our system on the Zynq 7000 XC7Z100 FPGA platform. Our design can prune 51 percent of the operations without any accuracy loss. Meanwhile, the energy efficiency of our system is as high as 6.55 × 105 Img/kJ, which is 118× better than the best accelerator based on an NVDIA Tesla-V100 GPU and 3.6× higher than the state-of-the-art FPGA implementations for BNNs.

scholarly journals Sentiment Classification Using Convolutional Neural Networks

2019 ◽  
Vol 9 (11) ◽  
pp. 2347 ◽  
Author(s):  
Hannah Kim ◽  
Young-Seob Jeong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Learning Models ◽  
Text Data ◽  
Textual Data ◽  
Better Than

As the number of textual data is exponentially increasing, it becomes more important to develop models to analyze the text data automatically. The texts may contain various labels such as gender, age, country, sentiment, and so forth. Using such labels may bring benefits to some industrial fields, so many studies of text classification have appeared. Recently, the Convolutional Neural Network (CNN) has been adopted for the task of text classification and has shown quite successful results. In this paper, we propose convolutional neural networks for the task of sentiment classification. Through experiments with three well-known datasets, we show that employing consecutive convolutional layers is effective for relatively longer texts, and our networks are better than other state-of-the-art deep learning models.

scholarly journals Energy-efficient memcapacitor devices for neuromorphic computing

2021 ◽  
Author(s):  
Kai-Uwe Demasius ◽  
Aron Kirschen ◽  
Stuart Parkin
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Neural Networks ◽  
Energy Efficiency ◽  
High Precision ◽  
Energy Efficient ◽  
Lateral Size ◽  
Data Intensive Computing ◽  
Data Intensive ◽  
Specialized Hardware

AbstractData-intensive computing operations, such as training neural networks, are essential for applications in artificial intelligence but are energy intensive. One solution is to develop specialized hardware onto which neural networks can be directly mapped, and arrays of memristive devices can, for example, be trained to enable parallel multiply–accumulate operations. Here we show that memcapacitive devices that exploit the principle of charge shielding can offer a highly energy-efficient approach for implementing parallel multiply–accumulate operations. We fabricate a crossbar array of 156 microscale memcapacitor devices and use it to train a neural network that could distinguish the letters ‘M’, ‘P’ and ‘I’. Modelling these arrays suggests that this approach could offer an energy efficiency of 29,600 tera-operations per second per watt, while ensuring high precision (6–8 bits). Simulations also show that the devices could potentially be scaled down to a lateral size of around 45 nm.

scholarly journals PSCNN: PatchShuffle Convolutional Neural Network for COVID-19 Explainable Diagnosis

2021 ◽  
Vol 9 ◽  
Author(s):  
Shui-Hua Wang ◽  
Ziquan Zhu ◽  
Yu-Dong Zhang
Keyword(s):  
Neural Network ◽  
Infectious Disease ◽  
Convolutional Neural Network ◽  
Loss Function ◽  
Data Augmentation ◽  
State Of The Art ◽  
Diagnosis System ◽  
Backbone Network ◽  
The Mean ◽  
Better Than

Objective: COVID-19 is a sort of infectious disease caused by a new strain of coronavirus. This study aims to develop a more accurate COVID-19 diagnosis system.Methods: First, the n-conv module (nCM) is introduced. Then we built a 12-layer convolutional neural network (12l-CNN) as the backbone network. Afterwards, PatchShuffle was introduced to integrate with 12l-CNN as a regularization term of the loss function. Our model was named PSCNN. Moreover, multiple-way data augmentation and Grad-CAM are employed to avoid overfitting and locating lung lesions.Results: The mean and standard variation values of the seven measures of our model were 95.28 ± 1.03 (sensitivity), 95.78 ± 0.87 (specificity), 95.76 ± 0.86 (precision), 95.53 ± 0.83 (accuracy), 95.52 ± 0.83 (F1 score), 91.7 ± 1.65 (MCC), and 95.52 ± 0.83 (FMI).Conclusion: Our PSCNN is better than 10 state-of-the-art models. Further, we validate the optimal hyperparameters in our model and demonstrate the effectiveness of PatchShuffle.

scholarly journals An Input-aware Factorization Machine for Sparse Prediction

2019 ◽  
Author(s):  
Yantao Yu ◽  
Zhen Wang ◽  
Bo Yuan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Real World ◽  
State Of The Art ◽  
Overall Performance ◽  
Factorization Machine ◽  
The Impact ◽  
Novel Model ◽  
Individual Input ◽  
Better Than

Factorization machines (FMs) are a class of general predictors working effectively with sparse data, which represents features using factorized parameters and weights. However, the accuracy of FMs can be adversely affected by the fixed representation trained for each feature, as the same feature is usually not equally predictive and useful in different instances. In fact, the inaccurate representation of features may even introduce noise and degrade the overall performance. In this work, we improve FMs by explicitly considering the impact of individual input upon the representation of features. We propose a novel model named \textit{Input-aware Factorization Machine} (IFM), which learns a unique input-aware factor for the same feature in different instances via a neural network. Comprehensive experiments on three real-world recommendation datasets are used to demonstrate the effectiveness and mechanism of IFM. Empirical results indicate that IFM is significantly better than the standard FM model and consistently outperforms four state-of-the-art deep learning based methods.

scholarly journals PSSPNN: PatchShuffle Stochastic Pooling Neural Network for an Explainable Diagnosis of COVID-19 with Multiple-Way Data Augmentation

2021 ◽  
Vol 2021 ◽  
pp. 1-18
Author(s):  
Shui-Hua Wang ◽  
Yin Zhang ◽  
Xiaochun Cheng ◽  
Xin Zhang ◽  
Yu-Dong Zhang
Keyword(s):  
Neural Network ◽  
Pulmonary Tuberculosis ◽  
Healthy Subjects ◽  
Data Augmentation ◽  
State Of The Art ◽  
Accurate Diagnosis ◽  
Test Set ◽  
The World ◽  
Stochastic Pooling ◽  
Better Than

Aim. COVID-19 has caused large death tolls all over the world. Accurate diagnosis is of significant importance for early treatment. Methods. In this study, we proposed a novel PSSPNN model for classification between COVID-19, secondary pulmonary tuberculosis, community-captured pneumonia, and healthy subjects. PSSPNN entails five improvements: we first proposed the n-conv stochastic pooling module. Second, a novel stochastic pooling neural network was proposed. Third, PatchShuffle was introduced as a regularization term. Fourth, an improved multiple-way data augmentation was used. Fifth, Grad-CAM was utilized to interpret our AI model. Results. The 10 runs with random seed on the test set showed our algorithm achieved a microaveraged F1 score of 95.79%. Moreover, our method is better than nine state-of-the-art approaches. Conclusion. This proposed PSSPNN will help assist radiologists to make diagnosis more quickly and accurately on COVID-19 cases.

A Neural Network for Image Anomaly Detection with Deep Pyramidal Representations and Dynamic Routing

2020 ◽  
Vol 30 (10) ◽  
pp. 2050060
Author(s):  
Pankaj Mishra ◽  
Claudio Piciarelli ◽  
Gian Luca Foresti
Keyword(s):  
Neural Network ◽  
Anomaly Detection ◽  
State Of The Art ◽  
Structural Similarity ◽  
Input Image ◽  
Image Features ◽  
Anomalous Data ◽  
Scale Levels ◽  
Anomaly Detector ◽  
Better Than

Image anomaly detection is an application-driven problem where the aim is to identify novel samples, which differ significantly from the normal ones. We here propose Pyramidal Image Anomaly DEtector (PIADE), a deep reconstruction-based pyramidal approach, in which image features are extracted at different scale levels to better catch the peculiarities that could help to discriminate between normal and anomalous data. The features are dynamically routed to a reconstruction layer and anomalies can be identified by comparing the input image with its reconstruction. Unlike similar approaches, the comparison is done by using structural similarity and perceptual loss rather than trivial pixel-by-pixel comparison. The proposed method performed at par or better than the state-of-the-art methods when tested on publicly available datasets such as CIFAR10, COIL-100 and MVTec.

scholarly journals ROBIN: A Robust Optical Binary Neural Network Accelerator

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-24
Author(s):  
Febin P. Sunny ◽  
Asif Mirza ◽  
Mahdi Nikdast ◽  
Sudeep Pasricha
Keyword(s):  
Neural Network ◽  
Energy Efficient ◽  
Network Models ◽  
Microring Resonator ◽  
Memory Consumption ◽  
Neural Network Models ◽  
Domain Specific ◽  
Binary Neural Network ◽  
Optical Domain ◽  
Time Overhead

Domain specific neural network accelerators have garnered attention because of their improved energy efficiency and inference performance compared to CPUs and GPUs. Such accelerators are thus well suited for resource-constrained embedded systems. However, mapping sophisticated neural network models on these accelerators still entails significant energy and memory consumption, along with high inference time overhead. Binarized neural networks (BNNs), which utilize single-bit weights, represent an efficient way to implement and deploy neural network models on accelerators. In this paper, we present a novel optical-domain BNN accelerator, named ROBIN , which intelligently integrates heterogeneous microring resonator optical devices with complementary capabilities to efficiently implement the key functionalities in BNNs. We perform detailed fabrication-process variation analyses at the optical device level, explore efficient corrective tuning for these devices, and integrate circuit-level optimization to counter thermal variations. As a result, our proposed ROBIN architecture possesses the desirable traits of being robust, energy-efficient, low latency, and high throughput, when executing BNN models. Our analysis shows that ROBIN can outperform the best-known optical BNN accelerators and many electronic accelerators. Specifically, our energy-efficient ROBIN design exhibits energy-per-bit values that are ∼4 × lower than electronic BNN accelerators and ∼933 × lower than a recently proposed photonic BNN accelerator, while a performance-efficient ROBIN design shows ∼3 × and ∼25 × better performance than electronic and photonic BNN accelerators, respectively.

Sign in / Sign up

Export Citation Format

Share Document