scholarly journals The Adversarial Attack and Detection under the Fisher Information Metric

2019 ◽  
Vol 33 ◽  
pp. 5869-5876 ◽  
Author(s):  
Chenxiao Zhao ◽  
P. Thomas Fletcher ◽  
Mixue Yu ◽  
Yaxin Peng ◽  
Guixu Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Fisher Information ◽  
Information Matrix ◽  
Superior Performance ◽  
Learning Models ◽  
Detection Algorithms ◽  
Fisher Information Metric ◽  
One Step ◽  
Information Metric ◽  
Adversarial Attack

Many deep learning models are vulnerable to the adversarial attack, i.e., imperceptible but intentionally-designed perturbations to the input can cause incorrect output of the networks. In this paper, using information geometry, we provide a reasonable explanation for the vulnerability of deep learning models. By considering the data space as a non-linear space with the Fisher information metric induced from a neural network, we first propose an adversarial attack algorithm termed one-step spectral attack (OSSA). The method is described by a constrained quadratic form of the Fisher information matrix, where the optimal adversarial perturbation is given by the first eigenvector, and the vulnerability is reflected by the eigenvalues. The larger an eigenvalue is, the more vulnerable the model is to be attacked by the corresponding eigenvector. Taking advantage of the property, we also propose an adversarial detection method with the eigenvalues serving as characteristics. Both our attack and detection algorithms are numerically optimized to work efficiently on large datasets. Our evaluations show superior performance compared with other methods, implying that the Fisher information is a promising approach to investigate the adversarial attacks and defenses.

scholarly journals Deep Malaria Parasite Detection in Thin Blood Smear Microscopic Images

2021 ◽  
Vol 11 (5) ◽  
pp. 2284
Author(s):  
Asma Maqsood ◽  
Muhammad Shahid Farid ◽  
Muhammad Hassan Khan ◽  
Marcin Grzegorzek
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Red Blood Cells ◽  
Convolutional Neural Networks ◽  
Blood Cells ◽  
Superior Performance ◽  
Learning Models ◽  
Infected Female ◽  
The World ◽  
Augmentation Techniques

Malaria is a disease activated by a type of microscopic parasite transmitted from infected female mosquito bites to humans. Malaria is a fatal disease that is endemic in many regions of the world. Quick diagnosis of this disease will be very valuable for patients, as traditional methods require tedious work for its detection. Recently, some automated methods have been proposed that exploit hand-crafted feature extraction techniques however, their accuracies are not reliable. Deep learning approaches modernize the world with their superior performance. Convolutional Neural Networks (CNN) are vastly scalable for image classification tasks that extract features through hidden layers of the model without any handcrafting. The detection of malaria-infected red blood cells from segmented microscopic blood images using convolutional neural networks can assist in quick diagnosis, and this will be useful for regions with fewer healthcare experts. The contributions of this paper are two-fold. First, we evaluate the performance of different existing deep learning models for efficient malaria detection. Second, we propose a customized CNN model that outperforms all observed deep learning models. It exploits the bilateral filtering and image augmentation techniques for highlighting features of red blood cells before training the model. Due to image augmentation techniques, the customized CNN model is generalized and avoids over-fitting. All experimental evaluations are performed on the benchmark NIH Malaria Dataset, and the results reveal that the proposed algorithm is 96.82% accurate in detecting malaria from the microscopic blood smears.

scholarly journals Short-Term Forecasting of Photovoltaic Solar Power Production Using Variational Auto-Encoder Driven Deep Learning Approach

2020 ◽  
Vol 10 (23) ◽  
pp. 8400 ◽  
Author(s):  
Abdelkader Dairi ◽  
Fouzi Harrou ◽  
Ying Sun ◽  
Sofiane Khadraoui
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Solar Power ◽  
Power Production ◽  
Superior Performance ◽  
Support Vector ◽  
Learning Models ◽  
Short Term ◽  
Learning Methods ◽  
Short Term Forecasting

The accurate modeling and forecasting of the power output of photovoltaic (PV) systems are critical to efficiently managing their integration in smart grids, delivery, and storage. This paper intends to provide efficient short-term forecasting of solar power production using Variational AutoEncoder (VAE) model. Adopting the VAE-driven deep learning model is expected to improve forecasting accuracy because of its suitable performance in time-series modeling and flexible nonlinear approximation. Both single- and multi-step-ahead forecasts are investigated in this work. Data from two grid-connected plants (a 243 kW parking lot canopy array in the US and a 9 MW PV system in Algeria) are employed to show the investigated deep learning models’ performance. Specifically, the forecasting outputs of the proposed VAE-based forecasting method have been compared with seven deep learning methods, namely recurrent neural network, Long short-term memory (LSTM), Bidirectional LSTM, Convolutional LSTM network, Gated recurrent units, stacked autoencoder, and restricted Boltzmann machine, and two commonly used machine learning methods, namely logistic regression and support vector regression. The results of this investigation demonstrate the satisfying performance of deep learning techniques to forecast solar power and point out that the VAE consistently performed better than the other methods. Also, results confirmed the superior performance of deep learning models compared to the two considered baseline machine learning models.

scholarly journals Holographic Fisher information metric in Schrödinger spacetime

2021 ◽  
Vol 136 (11) ◽  
Author(s):  
H. Dimov ◽  
I. N. Iliev ◽  
M. Radomirov ◽  
R. C. Rashkov ◽  
T. Vetsov
Keyword(s):  
Fisher Information ◽  
Fisher Information Metric ◽  
Information Metric

scholarly journals Continuous Training and Deployment of Deep Learning Models

2021 ◽  
Author(s):  
Ioannis Prapas ◽  
Behrouz Derakhshan ◽  
Alireza Rezaei Mahdiraji ◽  
Volker Markl
Keyword(s):  
Deep Learning ◽  
Historical Data ◽  
State Of The Art ◽  
Streaming Data ◽  
Superior Performance ◽  
Learning Models ◽  
Model Quality ◽  
Continuous Training ◽  
Training Time ◽  
Machine Learning Methods

AbstractDeep Learning (DL) has consistently surpassed other Machine Learning methods and achieved state-of-the-art performance in multiple cases. Several modern applications like financial and recommender systems require models that are constantly updated with fresh data. The prominent approach for keeping a DL model fresh is to trigger full retraining from scratch when enough new data are available. However, retraining large and complex DL models is time-consuming and compute-intensive. This makes full retraining costly, wasteful, and slow. In this paper, we present an approach to continuously train and deploy DL models. First, we enable continuous training through proactive training that combines samples of historical data with new streaming data. Second, we enable continuous deployment through gradient sparsification that allows us to send a small percentage of the model updates per training iteration. Our experimental results with LeNet5 on MNIST and modern DL models on CIFAR-10 show that proactive training keeps models fresh with comparable—if not superior—performance to full retraining at a fraction of the time. Combined with gradient sparsification, sparse proactive training enables very fast updates of a deployed model with arbitrarily large sparsity, reducing communication per iteration up to four orders of magnitude, with minimal—if any—losses in model quality. Sparse training, however, comes at a price; it incurs overhead on the training that depends on the size of the model and increases the training time by factors ranging from 1.25 to 3 in our experiments. Arguably, a small price to pay for successfully enabling the continuous training and deployment of large DL models.

Assessment of Deep Learning Models for Human Activity Recognition on Multi-variate Time Series Data and Non-targeted Adversarial Attack

2021 ◽  
pp. 129-159
Author(s):  
Mahbuba Tasmin ◽  
Sharif Uddin Ruman ◽  
Taoseef Ishtiak ◽  
Arif-ur-Rahman Chowdhury Suhan ◽  
Redwan Hasif ◽  
...  
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Activity Recognition ◽  
Human Activity ◽  
Time Series Data ◽  
Human Activity Recognition ◽  
Series Data ◽  
Learning Models ◽  
Adversarial Attack

scholarly journals Semantic-Preserving Adversarial Text Attacks

2021 ◽  
Author(s):  
Xinghao Yang ◽  
Yongshun Gong ◽  
Weifeng Liu ◽  
JAMES BAILEY ◽  
Tianqing Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Potential Candidate ◽  
Success Rates ◽  
Learning Models ◽  
Text Documents ◽  
Word Level ◽  
Sentence Level ◽  
Adversarial Attack ◽  
Semantic Preservation ◽  
Adversarial Example

Deep learning models are known immensely brittle to adversarial image examples, yet their vulnerability in text classification is insufficiently explored. Existing text adversarial attack strategies can be roughly divided into three categories, i.e., character-level attack, word-level attack, and sentence-level attack. Despite the success brought by recent text attack methods, how to induce misclassification with the minimal text modifications while keeping the lexical correctness, syntactic soundness, and semantic consistency simultaneously is still a challenge. To examine the vulnerability of deep models, we devise a Bigram and Unigram based adaptive Semantic Preservation Optimization (BU-SPO) approach which attacks text documents not only at a unigram word level but also at a bigram level to avoid generating meaningless sentences. We also present a hybrid attack strategy that collects substitution words from both synonyms and sememe candidates, to enrich the potential candidate set. Besides, a Semantic Preservation Optimization (SPO) method is devised to determine the word substitution priority and reduce the perturbation cost. Furthermore, we constraint the SPO with a semantic Filter (dubbed SPOF) to improve the semantic similarity between the input text and the adversarial example. To estimate the effectiveness of our proposed methods, BU-SPO and BU-SPOF, we attack four victim deep learning models trained on three real-world text datasets. Experimental results demonstrate that our approaches accomplish the highest semantics consistency and attack success rates by making the minimal word modifications compared with competitive methods.

scholarly journals Semantic-Preserving Adversarial Text Attacks

2021 ◽  
Author(s):  
Xinghao Yang ◽  
Yongshun Gong ◽  
Weifeng Liu ◽  
JAMES BAILEY ◽  
Tianqing Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Potential Candidate ◽  
Success Rates ◽  
Learning Models ◽  
Text Documents ◽  
Word Level ◽  
Sentence Level ◽  
Adversarial Attack ◽  
Semantic Preservation ◽  
Adversarial Example

Deep learning models are known immensely brittle to adversarial image examples, yet their vulnerability in text classification is insufficiently explored. Existing text adversarial attack strategies can be roughly divided into three categories, i.e., character-level attack, word-level attack, and sentence-level attack. Despite the success brought by recent text attack methods, how to induce misclassification with the minimal text modifications while keeping the lexical correctness, syntactic soundness, and semantic consistency simultaneously is still a challenge. To examine the vulnerability of deep models, we devise a Bigram and Unigram based adaptive Semantic Preservation Optimization (BU-SPO) approach which attacks text documents not only at a unigram word level but also at a bigram level to avoid generating meaningless sentences. We also present a hybrid attack strategy that collects substitution words from both synonyms and sememe candidates, to enrich the potential candidate set. Besides, a Semantic Preservation Optimization (SPO) method is devised to determine the word substitution priority and reduce the perturbation cost. Furthermore, we constraint the SPO with a semantic Filter (dubbed SPOF) to improve the semantic similarity between the input text and the adversarial example. To estimate the effectiveness of our proposed methods, BU-SPO and BU-SPOF, we attack four victim deep learning models trained on three real-world text datasets. Experimental results demonstrate that our approaches accomplish the highest semantics consistency and attack success rates by making the minimal word modifications compared with competitive methods.

Sign in / Sign up

Export Citation Format

Share Document