scholarly journals Application of recurrent and deep neural networks in classification tasks

2020 ◽  
Vol 20 (3) ◽  
pp. 59-79
Author(s):  
Lidio Mauro Lima De Campos ◽  
Danilo Souza Duarte
Keyword(s):  
Neural Networks ◽  
Multilayer Perceptron ◽  
Deep Neural Networks ◽  
Classification Tasks

As Redes Neurais Artificiais (RNAs) tem sido utilizadas nas soluções de variados problemas, dentre eles, os que envolvem tomada de decisões. Neste escopo, o objetivo desta pesquisa é apresentar uma ferramenta que dê suporte ao processo de decisão para seleção de cultivares de vinho e avaliação de carros, por meio da utilização de RNAs multilayer perceptron, profundas e recorrentes. Verificando-se sua eficácia e a melhor convergência, por meio do Modelo de Validação Cruzada. Os resultados elencados indicam a eficiência da técnica, para ambos os problemas, haja vista que a capacidade de generalização das RNAs testadas para o dataset wine foi em média de 85,58% utilizando a arquitetura de 3 camadas, 86,58% para a rede profunda e 93,53% para a rede recorrente, e para o dataset car evaluation  foi em média de 93,71% utilizando a rede recorrente.

scholarly journals Deep Scattering Spectra with Deep Neural Networks for Acoustic Scene Classification Tasks

2019 ◽  
Vol 28 (6) ◽  
pp. 1177-1183
Author(s):  
Pengyuan Zhang ◽  
Hangting Chen ◽  
Haichuan Bai ◽  
Qingsheng Yuan
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Scene Classification ◽  
Scattering Spectra ◽  
Classification Tasks

scholarly journals Inter-Class Angular Loss for Convolutional Neural Networks

2019 ◽  
Vol 33 ◽  
pp. 3894-3901 ◽  
Author(s):  
Le Hui ◽  
Xiang Li ◽  
Chen Gong ◽  
Meng Fang ◽  
Joey Tianyi Zhou ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
Learning Difficulties ◽  
Feature Space ◽  
Superior Performance ◽  
Strongly Correlated ◽  
Discriminative Ability ◽  
Practical Applications ◽  
Classification Tasks

Convolutional Neural Networks (CNNs) have shown great power in various classification tasks and have achieved remarkable results in practical applications. However, the distinct learning difficulties in discriminating different pairs of classes are largely ignored by the existing networks. For instance, in CIFAR-10 dataset, distinguishing cats from dogs is usually harder than distinguishing horses from ships. By carefully studying the behavior of CNN models in the training process, we observe that the confusion level of two classes is strongly correlated with their angular separability in the feature space. That is, the larger the inter-class angle is, the lower the confusion will be. Based on this observation, we propose a novel loss function dubbed “Inter-Class Angular Loss” (ICAL), which explicitly models the class correlation and can be directly applied to many existing deep networks. By minimizing the proposed ICAL, the networks can effectively discriminate the examples in similar classes by enlarging the angle between their corresponding class vectors. Thorough experimental results on a series of vision and nonvision datasets confirm that ICAL critically improves the discriminative ability of various representative deep neural networks and generates superior performance to the original networks with conventional softmax loss.

scholarly journals Universal adversarial attacks on deep neural networks for medical image classification

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Hokuto Hirano ◽  
Akinori Minagi ◽  
Kazuhiro Takemoto
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Clinical Diagnosis ◽  
Medical Image ◽  
Deep Neural Networks ◽  
Careful Consideration ◽  
Specific Class ◽  
High Stake ◽  
Classification Tasks ◽  
Medical Image Classification

Abstract Background Deep neural networks (DNNs) are widely investigated in medical image classification to achieve automated support for clinical diagnosis. It is necessary to evaluate the robustness of medical DNN tasks against adversarial attacks, as high-stake decision-making will be made based on the diagnosis. Several previous studies have considered simple adversarial attacks. However, the vulnerability of DNNs to more realistic and higher risk attacks, such as universal adversarial perturbation (UAP), which is a single perturbation that can induce DNN failure in most classification tasks has not been evaluated yet. Methods We focus on three representative DNN-based medical image classification tasks (i.e., skin cancer, referable diabetic retinopathy, and pneumonia classifications) and investigate their vulnerability to the seven model architectures of UAPs. Results We demonstrate that DNNs are vulnerable to both nontargeted UAPs, which cause a task failure resulting in an input being assigned an incorrect class, and to targeted UAPs, which cause the DNN to classify an input into a specific class. The almost imperceptible UAPs achieved > 80% success rates for nontargeted and targeted attacks. The vulnerability to UAPs depended very little on the model architecture. Moreover, we discovered that adversarial retraining, which is known to be an effective method for adversarial defenses, increased DNNs’ robustness against UAPs in only very few cases. Conclusion Unlike previous assumptions, the results indicate that DNN-based clinical diagnosis is easier to deceive because of adversarial attacks. Adversaries can cause failed diagnoses at lower costs (e.g., without consideration of data distribution); moreover, they can affect the diagnosis. The effects of adversarial defenses may not be limited. Our findings emphasize that more careful consideration is required in developing DNNs for medical imaging and their practical applications.

scholarly journals RALR: Random Amplify Learning Rates for Training Neural Networks

2021 ◽  
Vol 12 (1) ◽  
pp. 268
Author(s):  
Jiali Deng ◽  
Haigang Gong ◽  
Minghui Liu ◽  
Tianshu Xie ◽  
Xuan Cheng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Data Augmentation ◽  
Saddle Points ◽  
Learning Rate ◽  
Global Learning ◽  
Fine Grained ◽  
Learning Rates ◽  
Overall Performance ◽  
Classification Tasks

It has been shown that the learning rate is one of the most critical hyper-parameters for the overall performance of deep neural networks. In this paper, we propose a new method for setting the global learning rate, named random amplify learning rates (RALR), to improve the performance of any optimizer in training deep neural networks. Instead of monotonically decreasing the learning rate, we expect to escape saddle points or local minima by amplifying the learning rate between reasonable boundary values based on a given probability. Training with RALR rather than conventionally decreasing the learning rate achieves further improvement on networks’ performance without extra consumption. Remarkably, the RALR is complementary with state-of-the-art data augmentation and regularization methods. Besides, we empirically study its performance on image classification tasks, fine-grained classification tasks, object detection tasks, and machine translation tasks. Experiments demonstrate that RALR can bring a notable improvement while preventing overfitting when training deep neural networks. For example, the classification accuracy of ResNet-110 trained on the CIFAR-100 dataset using RALR achieves a 1.34% gain compared with ResNet-110 trained traditionally.

scholarly journals Simple Iterative Method for Generating Targeted Universal Adversarial Perturbations

Algorithms ◽  
2020 ◽  
Vol 13 (11) ◽  
pp. 268 ◽  
Author(s):  
Hokuto Hirano ◽  
Kazuhiro Takemoto
Keyword(s):  
Neural Networks ◽  
Iterative Method ◽  
Image Classification ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Specific Class ◽  
Targeted Attacks ◽  
Fast Gradient ◽  
Classification Tasks ◽  
Sign Method

Deep neural networks (DNNs) are vulnerable to adversarial attacks. In particular, a single perturbation known as the universal adversarial perturbation (UAP) can foil most classification tasks conducted by DNNs. Thus, different methods for generating UAPs are required to fully evaluate the vulnerability of DNNs. A realistic evaluation would be with cases that consider targeted attacks; wherein the generated UAP causes the DNN to classify an input into a specific class. However, the development of UAPs for targeted attacks has largely fallen behind that of UAPs for non-targeted attacks. Therefore, we propose a simple iterative method to generate UAPs for targeted attacks. Our method combines the simple iterative method for generating non-targeted UAPs and the fast gradient sign method for generating a targeted adversarial perturbation for an input. We applied the proposed method to state-of-the-art DNN models for image classification and proved the existence of almost imperceptible UAPs for targeted attacks; further, we demonstrated that such UAPs can be easily generated.

scholarly journals Regularizing Fully Convolutional Networks for Time Series Classification by Decorrelating Filters

2019 ◽  
Vol 33 ◽  
pp. 10003-10004 ◽  
Author(s):  
Kaushal Paneri ◽  
Vishnu TV ◽  
Pankaj Malhotra ◽  
Lovekesh Vig ◽  
Gautam Shroff
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Neural Networks ◽  
Training Data ◽  
Time Series Classification ◽  
Strong Correlations ◽  
Average Correlation ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
Classification Tasks

Deep neural networks are prone to overfitting, especially in small training data regimes. Often, these networks are overparameterized and the resulting learned weights tend to have strong correlations. However, convolutional networks in general, and fully convolution neural networks (FCNs) in particular, have been shown to be relatively parameter efficient, and have recently been successfully applied to time series classification tasks. In this paper, we investigate the application of different regularizers on the correlation between the learned convolutional filters in FCNs using Batch Normalization (BN) as a regularizer for time series classification (TSC) tasks. Results demonstrate that despite orthogonal initialization of the filters, the average correlation across filters (especially for filters in higher layers) tends to increase as training proceeds, indicating redundancy of filters. To mitigate this redundancy, we propose a strong regularizer, using simple yet effective filter decorrelation. Our proposed method yields significant gains in classification accuracy for 44 diverse time series datasets from the UCR TSC benchmark repository.

scholarly journals Disrupting Audio Event Detection Deep Neural Networks with White Noise

Technologies ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 64
Author(s):  
Rodrigo dos Santos ◽  
Ashwitha Kassetty ◽  
Shirin Nilizadeh
Keyword(s):  
Neural Networks ◽  
White Noise ◽  
Convolutional Neural Networks ◽  
Event Detection ◽  
Recurrent Neural Networks ◽  
Deep Neural Networks ◽  
Audio Event ◽  
Noise Disturbances ◽  
Classification Tasks ◽  
Percent Success

Audio event detection (AED) systems can leverage the power of specialized algorithms for detecting the presence of a specific sound of interest within audio captured from the environment. More recent approaches rely on deep learning algorithms, such as convolutional neural networks and convolutional recurrent neural networks. Given these conditions, it is important to assess how vulnerable these systems can be to attacks. As such, we develop AED-suited convolutional neural networks and convolutional recurrent neural networks, and attack them next with white noise disturbances, conceived to be simple and straightforward to be implemented and employed, even by non-tech savvy attackers. We develop this work under a safety-oriented scenario (AED systems for safety-related sounds, such as gunshots), and we show that an attacker can use such disturbances to avoid detection by up to 100 percent success. Prior work has shown that attackers can mislead image classification tasks; however, this work focuses on attacks against AED systems by tampering with their audio rather than image components. This work brings awareness to the designers and manufacturers of AED systems, as these solutions are vulnerable, yet may be trusted by individuals and families.

Sign in / Sign up

Export Citation Format

Share Document