RazorNet: Adversarial Training and Noise Training on a Deep Neural Network Fooled by a Shallow Neural Network

In this work, we propose ShallowDeepNet, a novel system architecture that includes a shallow and a deep neural network. The shallow neural network has the duty of data preprocessing and generating adversarial samples. The deep neural network has the duty of understanding data and information as well as detecting adversarial samples. The deep neural network gets its weights from transfer learning, adversarial training, and noise training. The system is examined on the biometric (fingerprint and iris) and the pharmaceutical data (pill image). According to the simulation results, the system is capable of improving the detection accuracy of the biometric data from 1.31% to 80.65% when the adversarial data is used and to 93.4% when the adversarial data as well as the noisy data are given to the network. The system performance on the pill image data is increased from 34.55% to 96.03% and then to 98.2%, respectively. Training on different types of noise can benefit us in detecting samples from unknown and unseen adversarial attacks. Meanwhile, the system training on the adversarial data as well as noisy data occurs only once. In fact, retraining the system may improve the performance further. Furthermore, training the system on new types of attacks and noise can help in enhancing the system performance.

Download Full-text

Harnessing the Adversarial Perturbation to Enhance Security in the Autoencoder-Based Communication System

Electronics ◽

10.3390/electronics9020294 ◽

2020 ◽

Vol 9 (2) ◽

pp. 294

Author(s):

Zhixiang Deng ◽

Qian Sang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Communication System ◽

Deep Neural Network ◽

The Self ◽

Full Duplex ◽

Training Method ◽

Adversarial Training ◽

Simulation Results ◽

Secure Transmission

Given the vulnerability of deep neural network to adversarial attacks, the application of deep learning in the wireless physical layer arouses comprehensive security concerns. In this paper, we consider an autoencoder-based communication system with a full-duplex (FD) legitimate receiver and an external eavesdropper. It is assumed that the system is trained from end-to-end based on the concepts of autoencoder. The FD legitimate receiver transmits a well-designed adversary perturbation signal to jam the eavesdropper while receiving information simultaneously. To defend the self-perturbation from the loop-back channel, the legitimate receiver is re-trained with the adversarial training method. The simulation results show that with the scheme proposed in this paper, the block-error-rate (BLER) of the legitimate receiver almost remains unaffected while the BLER of the eavesdropper is increased by orders of magnitude. This ensures reliable and secure transmission between the transmitter and the legitimate receiver.

Download Full-text

Anomaly Detection Using Deep Neural Network for IoT Architecture

Applied Sciences ◽

10.3390/app11157050 ◽

2021 ◽

Vol 11 (15) ◽

pp. 7050

Author(s):

Zeeshan Ahmad ◽

Adnan Shahid Khan ◽

Kashif Nisar ◽

Iram Haider ◽

Rosilah Hassan ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Anomaly Detection ◽

Deep Neural Network ◽

Detection System ◽

Traffic Monitoring ◽

Detection Accuracy ◽

Learning Models ◽

Prime Concern ◽

Entry Points

The revolutionary idea of the internet of things (IoT) architecture has gained enormous popularity over the last decade, resulting in an exponential growth in the IoT networks, connected devices, and the data processed therein. Since IoT devices generate and exchange sensitive data over the traditional internet, security has become a prime concern due to the generation of zero-day cyberattacks. A network-based intrusion detection system (NIDS) can provide the much-needed efficient security solution to the IoT network by protecting the network entry points through constant network traffic monitoring. Recent NIDS have a high false alarm rate (FAR) in detecting the anomalies, including the novel and zero-day anomalies. This paper proposes an efficient anomaly detection mechanism using mutual information (MI), considering a deep neural network (DNN) for an IoT network. A comparative analysis of different deep-learning models such as DNN, Convolutional Neural Network, Recurrent Neural Network, and its different variants, such as Gated Recurrent Unit and Long Short-term Memory is performed considering the IoT-Botnet 2020 dataset. Experimental results show the improvement of 0.57–2.6% in terms of the model’s accuracy, while at the same time reducing the FAR by 0.23–7.98% to show the effectiveness of the DNN-based NIDS model compared to the well-known deep learning models. It was also observed that using only the 16–35 best numerical features selected using MI instead of 80 features of the dataset result in almost negligible degradation in the model’s performance but helped in decreasing the overall model’s complexity. In addition, the overall accuracy of the DL-based models is further improved by almost 0.99–3.45% in terms of the detection accuracy considering only the top five categorical and numerical features.

Download Full-text

Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.5798 ◽

2020 ◽

Vol 34 (04) ◽

pp. 3858-3865

Author(s):

Huijie Feng ◽

Chunpeng Wu ◽

Guoyang Chen ◽

Weifeng Zhang ◽

Yang Ning

Keyword(s):

Neural Network ◽

High Probability ◽

Deep Neural Network ◽

State Of The Art ◽

Computationally Efficient ◽

Base Classifier ◽

Training Scheme ◽

Adversarial Training ◽

Gaussian Perturbation ◽

Probabilistic Robustness

Recently smoothing deep neural network based classifiers via isotropic Gaussian perturbation is shown to be an effective and scalable way to provide state-of-the-art probabilistic robustness guarantee against ℓ2 norm bounded adversarial perturbations. However, how to train a good base classifier that is accurate and robust when smoothed has not been fully investigated. In this work, we derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart when training the base classifier. It is computationally efficient and can be implemented in parallel with other empirical defense methods. We discuss how to implement it under both standard (non-adversarial) and adversarial training scheme. At the same time, we also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability. Our extensive experimentation demonstrates the effectiveness of the proposed training and certification approaches on CIFAR-10 and ImageNet datasets.

Download Full-text

Anomaly Detection of CAN Bus Messages Using A Deep Neural Network for Autonomous Vehicles

Applied Sciences ◽

10.3390/app9153174 ◽

2019 ◽

Vol 9 (15) ◽

pp. 3174 ◽

Cited By ~ 5

Author(s):

Zhou ◽

Li ◽

Shen

Keyword(s):

Neural Network ◽

Anomaly Detection ◽

Autonomous Vehicles ◽

Deep Neural Network ◽

Can Bus ◽

Detection Accuracy ◽

Area Network ◽

Data Packets ◽

Essential Components ◽

Triplet Loss

The in-vehicle controller area network (CAN) bus is one of the essential components for autonomous vehicles, and its safety will be one of the greatest challenges in the field of intelligent vehicles in the future. In this paper, we propose a novel system that uses a deep neural network (DNN) to detect anomalous CAN bus messages. We treat anomaly detection as a cross-domain modelling problem, in which three CAN bus data packets as a group are directly imported into the DNN architecture for parallel training with shared weights. After that, three data packets are represented as three independent feature vectors, which corresponds to three different types of data sequences, namely anchor, positive and negative. The proposed DNN architecture is an embedded triplet loss network that optimizes the distance between the anchor example and the positive example, makes it smaller than the distance between the anchor example and the negative example, and realizes the similarity calculation of samples, which were originally used in face detection. Compared to traditional anomaly detection methods, the proposed method to learn the parameters with shared-weight could improve detection efficiency and detection accuracy. The whole detection system is composed of the front-end and the back-end, which correspond to deep network and triplet loss network, respectively, and are trainable in an end-to-end fashion. Experimental results demonstrate that the proposed technology can make real-time responses to anomalies and attacks to the CAN bus, and significantly improve the detection ratio. To the best of our knowledge, the proposed method is the first used for anomaly detection in the in-vehicle CAN bus.

Download Full-text

Transfer Learning Using Rotated Image Data to Improve Deep Neural Network Performance

Lecture Notes in Computer Science - Image Analysis and Recognition ◽

10.1007/978-3-319-11758-4_32 ◽

2014 ◽

pp. 290-300 ◽

Cited By ~ 3

Author(s):

Telmo Amaral ◽

Luís M. Silva ◽

Luís A. Alexandre ◽

Chetak Kandaswamy ◽

Joaquim Marques de Sá ◽

...

Keyword(s):

Neural Network ◽

Transfer Learning ◽

Network Performance ◽

Deep Neural Network ◽

Image Data

Download Full-text

Super resolution DOA estimation based on deep neural network

Scientific Reports ◽

10.1038/s41598-020-76608-y ◽

2020 ◽

Vol 10 (1) ◽

Author(s):

Wanli Liu

Keyword(s):

Neural Network ◽

Potential Application ◽

Deep Neural Network ◽

State Of The Art ◽

Signal To Noise Ratio ◽

Super Resolution ◽

Doa Estimation ◽

Angular Distance ◽

Signal To Noise ◽

Simulation Results

AbstractRecently, deep neural network (DNN) studies on direction-of-arrival (DOA) estimations have attracted more and more attention. This new method gives an alternative way to deal with DOA problem and has successfully shown its potential application. However, these works are often restricted to previously known signal number, same signal-to-noise ratio (SNR) or large intersignal angular distance, which will hinder their generalization in real application. In this paper, we present a novel DNN framework that realizes higher resolution and better generalization to random signal number and SNR. Simulation results outperform that of previous works and reach the state of the art.

Download Full-text

CNN-Based Illumination Estimation with Semantic Information

Applied Sciences ◽

10.3390/app10144806 ◽

2020 ◽

Vol 10 (14) ◽

pp. 4806 ◽

Cited By ~ 1

Author(s):

Ho-Hyoung Choi ◽

Hyun-Soo Kang ◽

Byoung-Ju Yun

Keyword(s):

Neural Network ◽

Computer Vision ◽

Semantic Information ◽

Color Constancy ◽

Noisy Data ◽

Image Data ◽

Input Image ◽

The Novel ◽

Image Patches ◽

Wide Range

For more than a decade, both academia and industry have focused attention on the computer vision and in particular the computational color constancy (CVCC). The CVCC is used as a fundamental preprocessing task in a wide range of computer vision applications. While our human visual system (HVS) has the innate ability to perceive constant surface colors of objects under varying illumination spectra, the computer vision is facing the color constancy challenge in nature. Accordingly, this article proposes novel convolutional neural network (CNN) architecture based on the residual neural network which consists of pre-activation, atrous or dilated convolution and batch normalization. The proposed network can automatically decide what to learn from input image data and how to pool without supervision. When receiving input image data, the proposed network crops each image into image patches prior to training. Once the network begins learning, local semantic information is automatically extracted from the image patches and fed to its novel pooling layer. As a result of the semantic pooling, a weighted map or a mask is generated. Simultaneously, the extracted information is estimated and combined to form global information during training. The use of the novel pooling layer enables the proposed network to distinguish between useful data and noisy data, and thus efficiently remove noisy data during learning and evaluating. The main contribution of the proposed network is taking CVCC to higher accuracy and efficiency by adopting the novel pooling method. The experimental results demonstrate that the proposed network outperforms its conventional counterparts in estimation accuracy.

Download Full-text