Improving the robustness of binarized neural network using the EFAT method

2021 ◽  
pp. 14-23
Author(s):  
Trinh Quang Kien
Keyword(s):  
Neural Network ◽  
Network Inference ◽  
Binary Representation ◽  
Linear Transformations ◽  
Training Time ◽  
Practical Applications ◽  
Adversarial Attack ◽  
Projected Gradient Descent ◽  
The Impact ◽  
Attack Models

In recent years with the explosion of research in artificial intelligence, deep learning models based on convolutional neural networks (CNNs) are one of the promising architectures for practical applications thanks to their reasonably good achievable accuracy. However, CNNs characterized by convolutional layers often have a large number of parameters and computational workload, leading to large energy consumption for training and network inference. The binarized neural network (BNN) model has been recently proposed to overcome that drawback. The BNNs use binary representation for the inputs and weights, which inherently reduces memory requirements and simplifies computations while still maintaining acceptable accuracy. BNN thereby is very suited for the practical realization of Edge-AI application on resource- and energy-constrained devices such as embedded or mobile devices. As CNN and BNN both compose linear transformations layers,  they can be fooled by adversarial attack patterns. This topic has been actively studied recently but most of them are for CNN. In this work, we examine the impact of the adversarial attack on BNNs and propose a solution to improve the accuracy of BNN against this type of attack. Specifically, we use an Enhanced Fast Adversarial Training (EFAT) method to train the network that helps the BNN be more robust against major adversarial attack models with a very short training time. Experimental results with Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) attack models on our trained BNN network with MNIST dataset increased accuracy from 31.34% and 0.18% to 96.96% and 85.08%, respectively.

scholarly journals Accelerating Neural Network Inference on FPGA-Based Platforms—A Survey

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1025
Author(s):  
Ran Wu ◽  
Xinmin Guo ◽  
Jian Du ◽  
Junbao Li
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Network Inference ◽  
Semantic Segmentation ◽  
Object Identification ◽  
Data Reuse ◽  
Practical Applications ◽  
Resource Limited ◽  
Video Recognition ◽  
Future Work

The breakthrough of deep learning has started a technological revolution in various areas such as object identification, image/video recognition and semantic segmentation. Neural network, which is one of representative applications of deep learning, has been widely used and developed many efficient models. However, the edge implementation of neural network inference is restricted because of conflicts between the high computation and storage complexity and resource-limited hardware platforms in applications scenarios. In this paper, we research neural networks which are involved in the acceleration on FPGA-based platforms. The architecture of networks and characteristics of FPGA are analyzed, compared and summarized, as well as their influence on acceleration tasks. Based on the analysis, we generalize the acceleration strategies into five aspects—computing complexity, computing parallelism, data reuse, pruning and quantization. Then previous works on neural network acceleration are introduced following these topics. We summarize how to design a technical route for practical applications based on these strategies. Challenges in the path are discussed to provide guidance for future work.

scholarly journals Evaluating the Impact of Optical Interconnects on a Multi-Chip Machine-Learning Architecture

Electronics ◽  
2018 ◽  
Vol 7 (8) ◽  
pp. 130 ◽  
Author(s):  
Yuhwan Ro ◽  
Eojin Lee ◽  
Jung Ahn
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Optical Interconnects ◽  
Performance Model ◽  
Training Time ◽  
Performance Improvements ◽  
Cluster Architecture ◽  
The Neural Network ◽  
The Impact

Following trends that emphasize neural networks for machine learning, many studies regarding computing systems have focused on accelerating deep neural networks. These studies often propose utilizing the accelerator specialized in a neural network and the cluster architecture composed of interconnected accelerator chips. We observed that inter-accelerator communication within a cluster has a significant impact on the training time of the neural network. In this paper, we show the advantages of optical interconnects for multi-chip machine-learning architecture by demonstrating performance improvements through replacing electrical interconnects with optical ones in an existing multi-chip system. We propose to use highly practical optical interconnect implementation and devise an arithmetic performance model to fairly assess the impact of optical interconnects on a machine-learning accelerator platform. In our evaluation of nine Convolutional Neural Networks with various input sizes, 100 and 400 Gbps optical interconnects reduce the training time by an average of 20.6% and 35.6%, respectively, compared to the baseline system with 25.6 Gbps electrical ones.

scholarly journals A framework for the fine-grained evaluation of the instantaneous expected value of soccer possessions

2021 ◽  
Author(s):  
Javier Fernández ◽  
Luke Bornn ◽  
Daniel Cervone
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Comprehensive Analysis ◽  
Network Architectures ◽  
Tracking Data ◽  
Analysis Framework ◽  
Practical Applications ◽  
Fine Grained ◽  
Expected Outcome ◽  
The Impact

AbstractThe expected possession value (EPV) of a soccer possession represents the likelihood of a team scoring or conceding the next goal at any time instance. In this work, we develop a comprehensive analysis framework for the EPV, providing soccer practitioners with the ability to evaluate the impact of observed and potential actions, both visually and analytically. The EPV expression is decomposed into a series of subcomponents that model the influence of passes, ball drives and shot actions on the expected outcome of a possession. We show we can learn from spatiotemporal tracking data and obtain calibrated models for all the components of the EPV. For the components related with passes, we produce visually-interpretable probability surfaces from a series of deep neural network architectures built on top of flexible representations of game states. Additionally, we present a series of novel practical applications providing coaches with an enriched interpretation of specific game situations. This is, to our knowledge, the first EPV approach in soccer that uses this decomposition and incorporates the dynamics of the 22 players and the ball through tracking data.

Research on behavior recognition based on feature fusion of automatic coder and recurrent neural network

2020 ◽  
Vol 39 (6) ◽  
pp. 8927-8935
Author(s):  
Bing Zheng ◽  
Dawei Yun ◽  
Yan Liang
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Behavior Pattern ◽  
Rapid Development ◽  
Video Data ◽  
Support Vector ◽  
Behavior Recognition ◽  
Learning Methods ◽  
The Impact ◽  
Internet Of Things Technology

Under the impact of COVID-19, research on behavior recognition are highly needed. In this paper, we combine the algorithm of self-adaptive coder and recurrent neural network to realize the research of behavior pattern recognition. At present, most of the research of human behavior recognition is focused on the video data, which is based on the video number. At the same time, due to the complexity of video image data, it is easy to violate personal privacy. With the rapid development of Internet of things technology, it has attracted the attention of a large number of experts and scholars. Researchers have tried to use many machine learning methods, such as random forest, support vector machine and other shallow learning methods, which perform well in the laboratory environment, but there is still a long way to go from practical application. In this paper, a recursive neural network algorithm based on long and short term memory (LSTM) is proposed to realize the recognition of behavior patterns, so as to improve the accuracy of human activity behavior recognition.

European Economies Stability Faced With Potential Outburst of Sovereign Debt Crisis. An Empirical Study Using Neural Network

2013 ◽  
Vol 12 (2) ◽  
pp. 3255-3260
Author(s):  
Stelian Stancu ◽  
Alexandra Maria Constantin
Keyword(s):  
Neural Network ◽  
Sovereign Debt ◽  
Debt Crisis ◽  
Banking System ◽  
Sovereign Debt Crisis ◽  
European Level ◽  
Continuous Growth ◽  
Economic Context ◽  
European Economies ◽  
The Impact

Instilment, on a European level, of a state incompatible with the state of stability on a macroeconomic level and in the financial-banking system lead to continuous growth of vulnerability of European economies, situated at the verge of an outburst of sovereign debt crises. In this context, the current papers main objective is to produce a study regarding the vulnerability of European economies faced with potential outburst of sovereign debt crisis, which implies quantitative analysis of the impact of sovereign debt on the sensitivity of the European Unions economies. The paper also entails the following specific objectives: completing an introduction in the current European economic context, conceptualization of the notion of “sovereign debt crisis, presenting the methodology and obtained empirical results, as well as exposition of the conclusions.

scholarly journals Automated Ventricular System Segmentation in Paediatric Patients Treated for Hydrocephalus Using Deep Learning Methods

2019 ◽  
Vol 2019 ◽  
pp. 1-9 ◽  
Author(s):  
Michał Klimont ◽  
Mateusz Flieger ◽  
Jacek Rzeszutek ◽  
Joanna Stachera ◽  
Aleksandra Zakrzewska ◽  
...  
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Ct Images ◽  
Policy Transfer ◽  
Training Data ◽  
Intraobserver Variability ◽  
Practical Applications ◽  
Brain Scans ◽  
Rate Policy ◽  
Ct Brain

Hydrocephalus is a common neurological condition that can have traumatic ramifications and can be lethal without treatment. Nowadays, during therapy radiologists have to spend a vast amount of time assessing the volume of cerebrospinal fluid (CSF) by manual segmentation on Computed Tomography (CT) images. Further, some of the segmentations are prone to radiologist bias and high intraobserver variability. To improve this, researchers are exploring methods to automate the process, which would enable faster and more unbiased results. In this study, we propose the application of U-Net convolutional neural network in order to automatically segment CT brain scans for location of CSF. U-Net is a neural network that has proven to be successful for various interdisciplinary segmentation tasks. We optimised training using state of the art methods, including “1cycle” learning rate policy, transfer learning, generalized dice loss function, mixed float precision, self-attention, and data augmentation. Even though the study was performed using a limited amount of data (80 CT images), our experiment has shown near human-level performance. We managed to achieve a 0.917 mean dice score with 0.0352 standard deviation on cross validation across the training data and a 0.9506 mean dice score on a separate test set. To our knowledge, these results are better than any known method for CSF segmentation in hydrocephalic patients, and thus, it is promising for potential practical applications.

scholarly journals An image denoising method based on BP neural network optimized by improved whale optimization algorithm

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Chunzhi Wang ◽  
Min Li ◽  
Ruoxi Wang ◽  
Han Yu ◽  
Shuping Wang
Keyword(s):  
Neural Network ◽  
Image Denoising ◽  
Optimization Algorithm ◽  
Bp Neural Network ◽  
Weight Coefficient ◽  
Whale Optimization Algorithm ◽  
Denoising Method ◽  
Training Time ◽  
Whale Optimization ◽  
City Construction

AbstractAs an important part of smart city construction, traffic image denoising has been studied widely. Image denoising technique can enhance the performance of segmentation and recognition model and improve the accuracy of segmentation and recognition results. However, due to the different types of noise and the degree of noise pollution, the traditional image denoising methods generally have some problems, such as blurred edges and details, loss of image information. This paper presents an image denoising method based on BP neural network optimized by improved whale optimization algorithm. Firstly, the nonlinear convergence factor and adaptive weight coefficient are introduced into the algorithm to improve the optimization ability and convergence characteristics of the standard whale optimization algorithm. Then, the improved whale optimization algorithm is used to optimize the initial weight and threshold value of BP neural network to overcome the dependence in the construction process, and shorten the training time of the neural network. Finally, the optimized BP neural network is applied to benchmark image denoising and traffic image denoising. The experimental results show that compared with the traditional denoising methods such as Median filtering, Neighborhood average filtering and Wiener filtering, the proposed method has better performance in peak signal-to-noise ratio.

Security Threat Analyses and Attack Models for Approximate Computing Systems

2021 ◽  
Vol 26 (4) ◽  
pp. 1-31
Author(s):  
Pruthvy Yellu ◽  
Landon Buell ◽  
Miguel Mark ◽  
Michel A. Kinsy ◽  
Dongpeng Xu ◽  
...  
Keyword(s):  
Error Resilience ◽  
Security Threats ◽  
Approximate Computing ◽  
Security Threat ◽  
Security Vulnerabilities ◽  
Computing Systems ◽  
Quantitative Analyses ◽  
Resilience Mechanisms ◽  
The Impact ◽  
Attack Models

Approximate computing (AC) represents a paradigm shift from conventional precise processing to inexact computation but still satisfying the system requirement on accuracy. The rapid progress on the development of diverse AC techniques allows us to apply approximate computing to many computation-intensive applications. However, the utilization of AC techniques could bring in new unique security threats to computing systems. This work does a survey on existing circuit-, architecture-, and compiler-level approximate mechanisms/algorithms, with special emphasis on potential security vulnerabilities. Qualitative and quantitative analyses are performed to assess the impact of the new security threats on AC systems. Moreover, this work proposes four unique visionary attack models, which systematically cover the attacks that build covert channels, compensate approximation errors, terminate normal error resilience mechanisms, and propagate additional errors. To thwart those attacks, this work further offers the guideline of countermeasure designs. Several case studies are provided to illustrate the implementation of the suggested countermeasures.

scholarly journals Artificial neural network, predictor variables and sensitivity threshold for DNA methylation-based age prediction using blood samples

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Zhonghui Thong ◽  
Jolena Ying Ying Tan ◽  
Eileen Shuzhen Loo ◽  
Yu Wei Phua ◽  
Xavier Liang Shun Chan ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Regression Model ◽  
Sensitivity Threshold ◽  
Ann Model ◽  
Blood Samples ◽  
Ann Models ◽  
Artificial Neural ◽  
The Impact ◽  
Age Prediction

AbstractRegression models are often used to predict age of an individual based on methylation patterns. Artificial neural network (ANN) however was recently shown to be more accurate for age prediction. Additionally, the impact of ethnicity and sex on our previous regression model have not been studied. Furthermore, there is currently no age prediction study investigating the lower limit of input DNA at the bisulfite treatment stage prior to pyrosequencing. Herein, we evaluated both regression and ANN models, and the impact of ethnicity and sex on age prediction for 333 local blood samples using three loci on the pyrosequencing platform. Subsequently, we trained a one locus-based ANN model to reduce the amount of DNA used. We demonstrated that the ANN model has a higher accuracy of age prediction than the regression model. Additionally, we showed that ethnicity did not affect age prediction among local Chinese, Malays and Indians. Although the predicted age of males were marginally overestimated, sex did not impact the accuracy of age prediction. Lastly, we present a one locus, dual CpG model using 25 ng of input DNA that is sufficient for forensic age prediction. In conclusion, the two ANN models validated would be useful for age prediction to provide forensic intelligence leads.

scholarly journals Graphene-based 3D XNOR-VRRAM with ternary precision for neuromorphic computing

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Batyrbek Alimkhanuly ◽  
Joon Sohn ◽  
Ik-Joon Chang ◽  
Seunghyun Lee
Keyword(s):  
Neural Network ◽  
Energy Consumption ◽  
Recognition Accuracy ◽  
Material Selection ◽  
Weighted Sum ◽  
Device Design ◽  
Key Factors ◽  
Neuromorphic Computing ◽  
Device Scaling ◽  
The Impact

AbstractRecent studies on neural network quantization have demonstrated a beneficial compromise between accuracy, computation rate, and architecture size. Implementing a 3D Vertical RRAM (VRRAM) array accompanied by device scaling may further improve such networks’ density and energy consumption. Individual device design, optimized interconnects, and careful material selection are key factors determining the overall computation performance. In this work, the impact of replacing conventional devices with microfabricated, graphene-based VRRAM is investigated for circuit and algorithmic levels. By exploiting a sub-nm thin 2D material, the VRRAM array demonstrates an improved read/write margins and read inaccuracy level for the weighted-sum procedure. Moreover, energy consumption is significantly reduced in array programming operations. Finally, an XNOR logic-inspired architecture designed to integrate 1-bit ternary precision synaptic weights into graphene-based VRRAM is introduced. Simulations on VRRAM with metal and graphene word-planes demonstrate 83.5 and 94.1% recognition accuracy, respectively, denoting the importance of material innovation in neuromorphic computing.

Sign in / Sign up

Export Citation Format

Share Document