Theoretical Investigation of Generalization Bounds for Adversarial Learning of Deep Neural Networks

2021 ◽  
Vol 15 (2) ◽  
Author(s):  
Qingyi Gao ◽  
Xiao Wang
Keyword(s):  
Neural Networks ◽  
Theoretical Investigation ◽  
Deep Neural Networks ◽  
Adversarial Learning ◽  
Generalization Bounds

scholarly journals Domain Generalization Using a Mixture of Multiple Latent Domains

2020 ◽  
Vol 34 (07) ◽  
pp. 11749-11756 ◽  
Author(s):  
Toshihiko Matsuura ◽  
Tatsuya Harada
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Multiple Source ◽  
Web Crawling ◽  
Not Given ◽  
Adversarial Learning ◽  
Generalized Model ◽  
Generalization Performance ◽  
Invariant Feature ◽  
Feature Extractor

When domains, which represent underlying data distributions, vary during training and testing processes, deep neural networks suffer a drop in their performance. Domain generalization allows improvements in the generalization performance for unseen target domains by using multiple source domains. Conventional methods assume that the domain to which each sample belongs is known in training. However, many datasets, such as those collected via web crawling, contain a mixture of multiple latent domains, in which the domain of each sample is unknown. This paper introduces domain generalization using a mixture of multiple latent domains as a novel and more realistic scenario, where we try to train a domain-generalized model without using domain labels. To address this scenario, we propose a method that iteratively divides samples into latent domains via clustering, and which trains the domain-invariant feature extractor shared among the divided latent domains via adversarial learning. We assume that the latent domain of images is reflected in their style, and thus, utilize style features for clustering. By using these features, our proposed method successfully discovers latent domains and achieves domain generalization even if the domain labels are not given. Experiments show that our proposed method can train a domain-generalized model without using domain labels. Moreover, it outperforms conventional domain generalization methods, including those that utilize domain labels.

scholarly journals Learning Sparse Neural Networks for Better Generalization

2020 ◽  
Author(s):  
Shiwei Liu
Keyword(s):  
Neural Networks ◽  
Test Data ◽  
Deep Neural Networks ◽  
Generalization Bounds ◽  
Better Than

Deep neural networks perform well on test data when they are highly overparameterized, which, however, also leads to large cost to train and deploy them. As a leading approach to address this problem, sparse neural networks have been widely used to significantly reduce the size of networks, making them more efficient during training and deployment, without compromising performance. Recently, sparse neural networks, either compressed from a pre-trained model or obtained by training from scratch, have been observed to be able to generalize as well as or even better than their dense counterparts. However, conventional techniques to find well fitted sparse sub-networks are expensive and the mechanisms underlying this phenomenon are far from clear. To tackle these problems, this Ph.D. research aims to study the generalization of sparse neural networks, and to propose more efficient approaches that can yield sparse neural networks with generalization bounds.

Deep neural networks trained with heavier data augmentation learn features closer to representations in hIT

2018 ◽  
Author(s):  
Alex Hernández-García ◽  
Johannes Mehrer ◽  
Nikolaus Kriegeskorte ◽  
Peter König ◽  
Tim C. Kietzmann
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Data Augmentation

Representation of adversarial images in deep neural networks and the human brain

2018 ◽  
Author(s):  
Chi Zhang ◽  
Xiaohan Duan ◽  
Ruyuan Zhang ◽  
Li Tong
Keyword(s):  
Neural Networks ◽  
Human Brain ◽  
Deep Neural Networks
Sign in / Sign up

Export Citation Format

Share Document