Improving the accuracy and robustness of RRAM-based in-memory computing against RRAM hardware noise and adversarial attacks

2021 ◽  
Author(s):  
Sai Kiran Cherupally ◽  
Jian Meng ◽  
Adnan Siraj Rakin ◽  
Shihui Yin ◽  
Injune Yeo ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Black Box ◽  
Accuracy Improvement ◽  
Training Scheme ◽  
Injection Scheme ◽  
Noise Injection ◽  
Noise Data ◽  
The Impact

Abstract We present a novel deep neural network (DNN) training scheme and RRAM in-memory computing (IMC) hardware evaluation towards achieving high robustness to the RRAM device/array variations and adversarial input attacks. We present improved IMC inference accuracy results evaluated on state-of-the-art DNNs including ResNet-18, AlexNet, and VGG with binary, 2-bit, and 4-bit activation/weight precision for the CIFAR-10 dataset. These DNNs are evaluated with measured noise data obtained from three different RRAM-based IMC prototype chips. Across these various DNNs and IMC chip measurements, we show that our proposed hardware noise-aware DNN training consistently improves DNN inference accuracy for actual IMC hardware, up to 8% accuracy improvement for the CIFAR-10 dataset. We also analyze the impact of our proposed noise injection scheme on the adversarial robustness of ResNet-18 DNNs with 1-bit, 2-bit, and 4-bit activation/weight precision. Our results show up to 6% improvement in the robustness to black-box adversarial input attacks.

scholarly journals Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

2020 ◽  
Vol 34 (04) ◽  
pp. 3858-3865
Author(s):  
Huijie Feng ◽  
Chunpeng Wu ◽  
Guoyang Chen ◽  
Weifeng Zhang ◽  
Yang Ning
Keyword(s):  
Neural Network ◽  
High Probability ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Computationally Efficient ◽  
Base Classifier ◽  
Training Scheme ◽  
Adversarial Training ◽  
Gaussian Perturbation ◽  
Probabilistic Robustness

Recently smoothing deep neural network based classifiers via isotropic Gaussian perturbation is shown to be an effective and scalable way to provide state-of-the-art probabilistic robustness guarantee against ℓ2 norm bounded adversarial perturbations. However, how to train a good base classifier that is accurate and robust when smoothed has not been fully investigated. In this work, we derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart when training the base classifier. It is computationally efficient and can be implemented in parallel with other empirical defense methods. We discuss how to implement it under both standard (non-adversarial) and adversarial training scheme. At the same time, we also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability. Our extensive experimentation demonstrates the effectiveness of the proposed training and certification approaches on CIFAR-10 and ImageNet datasets.

Validating the impact of accounting disclosures on stock market: A deep neural network approach

2021 ◽  
Vol 170 ◽  
pp. 120903
Author(s):  
Prajwal Eachempati ◽  
Praveen Ranjan Srivastava ◽  
Ajay Kumar ◽  
Kim Hua Tan ◽  
Shivam Gupta
Keyword(s):  
Neural Network ◽  
Stock Market ◽  
Deep Neural Network ◽  
Network Approach ◽  
Neural Network Approach ◽  
Accounting Disclosures ◽  
The Impact

scholarly journals Fast Accurate and Automatic Brushstroke Extraction

2021 ◽  
Vol 17 (2) ◽  
pp. 1-24
Author(s):  
Yunfei Fu ◽  
Hongchuan Yu ◽  
Chih-Kuo Yeh ◽  
Tong-Yee Lee ◽  
Jian J. Zhang
Keyword(s):  
Neural Network ◽  
Efficient Algorithm ◽  
Deep Neural Network ◽  
High Efficiency ◽  
State Of The Art ◽  
High Reliability ◽  
The Other ◽  
Manual Annotation ◽  
Stroke Extraction ◽  
Art Research

Brushstrokes are viewed as the artist’s “handwriting” in a painting. In many applications such as style learning and transfer, mimicking painting, and painting authentication, it is highly desired to quantitatively and accurately identify brushstroke characteristics from old masters’ pieces using computer programs. However, due to the nature of hundreds or thousands of intermingling brushstrokes in the painting, it still remains challenging. This article proposes an efficient algorithm for brush Stroke extraction based on a Deep neural network, i.e., DStroke. Compared to the state-of-the-art research, the main merit of the proposed DStroke is to automatically and rapidly extract brushstrokes from a painting without manual annotation, while accurately approximating the real brushstrokes with high reliability. Herein, recovering the faithful soft transitions between brushstrokes is often ignored by the other methods. In fact, the details of brushstrokes in a master piece of painting (e.g., shapes, colors, texture, overlaps) are highly desired by artists since they hold promise to enhance and extend the artists’ powers, just like microscopes extend biologists’ powers. To demonstrate the high efficiency of the proposed DStroke, we perform it on a set of real scans of paintings and a set of synthetic paintings, respectively. Experiments show that the proposed DStroke is noticeably faster and more accurate at identifying and extracting brushstrokes, outperforming the other methods.

scholarly journals An Efficient Method for Detection of DDoS Attacks on the Web Using Deep Learning Algorithms

2021 ◽  
Vol 10 (4) ◽  
pp. 2821-2829
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Ddos Attacks ◽  
Problem Statement ◽  
Neural Network Approach ◽  
Learning Techniques ◽  
Attack Data ◽  
Deep Learning Neural Network

Recently, DDoS attacks is the most significant threat in network security. Both industry and academia are currently debating how to detect and protect against DDoS attacks. Many studies are provided to detect these types of attacks. Deep learning techniques are the most suitable and efficient algorithm for categorizing normal and attack data. Hence, a deep neural network approach is proposed in this study to mitigate DDoS attacks effectively. We used a deep learning neural network to identify and classify traffic as benign or one of four different DDoS attacks. We will concentrate on four different DDoS types: Slowloris, Slowhttptest, DDoS Hulk, and GoldenEye. The rest of the paper is organized as follow: Firstly, we introduce the work, Section 2 defines the related works, Section 3 presents the problem statement, Section 4 describes the proposed methodology, Section 5 illustrate the results of the proposed methodology and shows how the proposed methodology outperforms state-of-the-art work and finally Section VI concludes the paper.

scholarly journals An Analysis of State-of-the-art Activation Functions For Supervised Deep Neural Network

2021 ◽  
Author(s):  
Anh Nguyen ◽  
Khoa Pham ◽  
Dat Ngo ◽  
Thanh Ngo ◽  
Lam Pham
Keyword(s):  
Neural Network ◽  
Supervised Classification ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Network Architectures ◽  
Activation Functions ◽  
Scene Classification ◽  
Learning Network ◽  
Deep Learning Network

This paper provides an analysis of state-of-the-art activation functions with respect to supervised classification of deep neural network. These activation functions comprise of Rectified Linear Units (ReLU), Exponential Linear Unit (ELU), Scaled Exponential Linear Unit (SELU), Gaussian Error Linear Unit (GELU), and the Inverse Square Root Linear Unit (ISRLU). To evaluate, experiments over two deep learning network architectures integrating these activation functions are conducted. The first model, basing on Multilayer Perceptron (MLP), is evaluated with MNIST dataset to perform these activation functions.Meanwhile, the second model, likely VGGish-based architecture, is applied for Acoustic Scene Classification (ASC) Task 1A in DCASE 2018 challenge, thus evaluate whether these activation functions work well in different datasets as well as different network architectures.

scholarly journals DANNP: an efficient artificial neural network pruning tool

2017 ◽  
Vol 3 ◽  
pp. e137 ◽  
Author(s):  
Mona Alshahrani ◽  
Othman Soufan ◽  
Arturo Magana-Mora ◽  
Vladimir B. Bajic
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Model Performance ◽  
Training Data ◽  
Classification Problems ◽  
Link Type ◽  
On Line ◽  
Pruning Algorithms ◽  
Artificial Neural ◽  
The Impact

Background Artificial neural networks (ANNs) are a robust class of machine learning models and are a frequent choice for solving classification problems. However, determining the structure of the ANNs is not trivial as a large number of weights (connection links) may lead to overfitting the training data. Although several ANN pruning algorithms have been proposed for the simplification of ANNs, these algorithms are not able to efficiently cope with intricate ANN structures required for complex classification problems. Methods We developed DANNP, a web-based tool, that implements parallelized versions of several ANN pruning algorithms. The DANNP tool uses a modified version of the Fast Compressed Neural Network software implemented in C++ to considerably enhance the running time of the ANN pruning algorithms we implemented. In addition to the performance evaluation of the pruned ANNs, we systematically compared the set of features that remained in the pruned ANN with those obtained by different state-of-the-art feature selection (FS) methods. Results Although the ANN pruning algorithms are not entirely parallelizable, DANNP was able to speed up the ANN pruning up to eight times on a 32-core machine, compared to the serial implementations. To assess the impact of the ANN pruning by DANNP tool, we used 16 datasets from different domains. In eight out of the 16 datasets, DANNP significantly reduced the number of weights by 70%–99%, while maintaining a competitive or better model performance compared to the unpruned ANN. Finally, we used a naïve Bayes classifier derived with the features selected as a byproduct of the ANN pruning and demonstrated that its accuracy is comparable to those obtained by the classifiers trained with the features selected by several state-of-the-art FS methods. The FS ranking methodology proposed in this study allows the users to identify the most discriminant features of the problem at hand. To the best of our knowledge, DANNP (publicly available at www.cbrc.kaust.edu.sa/dannp) is the only available and on-line accessible tool that provides multiple parallelized ANN pruning options. Datasets and DANNP code can be obtained at www.cbrc.kaust.edu.sa/dannp/data.php and https://doi.org/10.5281/zenodo.1001086.

scholarly journals Covid-19 detection via deep neural network and occlusion sensitivity maps

2021 ◽  
Author(s):  
Noor Ahmad ◽  
Muhammad Aminu ◽  
Mohd Halim Mohd Noor
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Color Images ◽  
Fine Tuning ◽  
Training Dataset ◽  
Learning Approaches ◽  
Learning Models ◽  
Sensitivity Maps

Deep learning approaches have attracted a lot of attention in the automatic detection of Covid-19 and transfer learning is the most common approach. However, majority of the pre-trained models are trained on color images, which can cause inefficiencies when fine-tuning the models on Covid-19 images which are often grayscale. To address this issue, we propose a deep learning architecture called CovidNet which requires a relatively smaller number of parameters. CovidNet accepts grayscale images as inputs and is suitable for training with limited training dataset. Experimental results show that CovidNet outperforms other state-of-the-art deep learning models for Covid-19 detection.

scholarly journals DeepLocate: Smartphone Based Indoor Localization with a Deep Neural Network Ensemble Classifier

Sensors ◽  
2019 ◽  
Vol 20 (1) ◽  
pp. 133 ◽  
Author(s):  
Imran Ashraf ◽  
Soojung Hur ◽  
Sangjoon Park ◽  
Yongwan Park
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Indoor Localization ◽  
Ensemble Classifier ◽  
Location Based Services ◽  
Magnetic Data ◽  
Neural Network Ensemble ◽  
Localization Accuracy ◽  
Mean Error ◽  
The Impact

A quickly growing location-based services area has led to increased demand for indoor positioning and localization. Undoubtedly, Wi-Fi fingerprint-based localization is one of the promising indoor localization techniques, yet the variation of received signal strength is a major problem for accurate localization. Magnetic field-based localization has emerged as a new player and proved a potential indoor localization technology. However, one of its major limitations is degradation in localization accuracy when various smartphones are used. The localization performance is different from various smartphones even with the same localization technique. This research leverages the use of a deep neural network-based ensemble classifier to perform indoor localization with heterogeneous devices. The chief aim is to devise an approach that can achieve a similar localization accuracy using various smartphones. Features extracted from magnetic data of Galaxy S8 are fed into neural networks (NNs) for training. The experiments are performed with Galaxy S8, LG G6, LG G7, and Galaxy A8 smartphones to investigate the impact of device dependence on localization accuracy. Results demonstrate that NNs can play a significant role in mitigating the impact of device heterogeneity and increasing indoor localization accuracy. The proposed approach is able to achieve a localization accuracy of 2.64 m at 50% on four different devices. The mean error is 2.23 m, 2.52 m, 2.59 m, and 2.78 m for Galaxy S8, LG G6, LG G7, and Galaxy A8, respectively. Experiments on a publicly available magnetic dataset of Sony Xperia M2 using the proposed approach show a mean error of 2.84 m with a standard deviation of 2.24 m, while the error at 50% is 2.33 m. Furthermore, the impact of devices on various attitudes on the localization accuracy is investigated.

Sign in / Sign up

Export Citation Format

Share Document