scholarly journals Simultaneously Improve Transferability and Discriminability for Adversarial Domain Adaptation

Entropy ◽  
2021 ◽  
Vol 24 (1) ◽  
pp. 44
Author(s):  
Ting Xiao ◽  
Cangning Fan ◽  
Peng Liu ◽  
Hongwei Liu
Keyword(s):  
Conditional Distribution ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Feature Space ◽  
Low Rank ◽  
Computationally Efficient ◽  
Adversarial Learning ◽  
Joint Distributions ◽  
Distribution Matching ◽  
Maximum Separation

Although adversarial domain adaptation enhances feature transferability, the feature discriminability will be degraded in the process of adversarial learning. Moreover, most domain adaptation methods only focus on distribution matching in the feature space; however, shifts in the joint distributions of input features and output labels linger in the network, and thus, the transferability is not fully exploited. In this paper, we propose a matrix rank embedding (MRE) method to enhance feature discriminability and transferability simultaneously. MRE restores a low-rank structure for data in the same class and enforces a maximum separation structure for data in different classes. In this manner, the variations within the subspace are reduced, and the separation between the subspaces is increased, resulting in improved discriminability. In addition to statistically aligning the class-conditional distribution in the feature space, MRE forces the data of the same class in different domains to exhibit an approximate low-rank structure, thereby aligning the class-conditional distribution in the label space, resulting in improved transferability. MRE is computationally efficient and can be used as a plug-and-play term for other adversarial domain adaptation networks. Comprehensive experiments demonstrate that MRE can advance state-of-the-art domain adaptation methods.

scholarly journals OVL: One-View Learning for Human Retrieval

2020 ◽  
Vol 34 (07) ◽  
pp. 11410-11417
Author(s):  
Wenjing Li ◽  
Zhongcheng Wu
Keyword(s):  
Large Scale ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Set Domain ◽  
Adversarial Learning ◽  
Learning Framework ◽  
Training Images ◽  
Open Set ◽  
Camera View ◽  
Supervised Methods

This paper considers a novel problem, named One-View Learning (OVL), in human retrieval a.k.a. person re-identification (re-ID). Unlike fully-supervised learning, OVL only requires pretty cheap annotation cost: labeled training images are only provided from one camera view (source view/domain), while the annotations of training images from other camera views (target views/domains) are not available. OVL is a problem of multi-target open set domain adaptation that is difficult for existing domain adaptation methods to handle. This is because 1) unlabeled samples are drawn from multiple target views in different distributions, and 2) the target views may contain samples of “unknown identity” that are not shared by the source view. To address this problem, this work introduces a novel one-view learning framework for person re-ID. This is achieved by adversarial multi-view learning (AMVL) and adversarial unknown rejection learning (AURL). The former learns a multi-view discriminator by adversarial learning to align the feature distributions between all views. The later is designed to reject unknown samples from target views through adversarial learning with two unknown identity classifiers. Extensive experiments on three large-scale datasets demonstrate the advantage of the proposed method over state-of-the-art domain adaptation and semi-supervised methods.

scholarly journals Discriminative Adversarial Domain Adaptation

2020 ◽  
Vol 34 (04) ◽  
pp. 5940-5947 ◽  
Author(s):  
Hui Tang ◽  
Kui Jia
Keyword(s):  
Domain Adaptation ◽  
Set Domain ◽  
Target Domain ◽  
Adversarial Learning ◽  
Closed Set ◽  
Joint Distributions ◽  
Open Set ◽  
Benchmark Datasets ◽  
Adversarial Training ◽  
Minimax Game

Given labeled instances on a source domain and unlabeled ones on a target domain, unsupervised domain adaptation aims to learn a task classifier that can well classify target instances. Recent advances rely on domain-adversarial training of deep networks to learn domain-invariant features. However, due to an issue of mode collapse induced by the separate design of task and domain classifiers, these methods are limited in aligning the joint distributions of feature and category across domains. To overcome it, we propose a novel adversarial learning method termed Discriminative Adversarial Domain Adaptation (DADA). Based on an integrated category and domain classifier, DADA has a novel adversarial objective that encourages a mutually inhibitory relation between category and domain predictions for any input instance. We show that under practical conditions, it defines a minimax game that can promote the joint distribution alignment. Except for the traditional closed set domain adaptation, we also extend DADA for extremely challenging problem settings of partial and open set domain adaptation. Experiments show the efficacy of our proposed methods and we achieve the new state of the art for all the three settings on benchmark datasets.

scholarly journals An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 12613-12620 ◽  
Author(s):  
Jihan Yang ◽  
Ruijia Xu ◽  
Ruiyu Li ◽  
Xiaojuan Qi ◽  
Xiaoyong Shen ◽  
...  
Keyword(s):  
Marginal Distribution ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Feature Space ◽  
Semantic Segmentation ◽  
The State ◽  
Object Size ◽  
Feature Maps ◽  
Feature Representations ◽  
Unsupervised Domain Adaptation

We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 → Cityscapes and SYNTHIA → Cityscapes.

scholarly journals Adversarial-Learned Loss for Domain Adaptation

2020 ◽  
Vol 34 (04) ◽  
pp. 3521-3528
Author(s):  
Minghao Chen ◽  
Shuai Zhao ◽  
Haifeng Liu ◽  
Deng Cai
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Confusion Matrix ◽  
Ground Truth ◽  
Adversarial Learning ◽  
Label Method ◽  
Model Predictions ◽  
Novel Method ◽  
Remarkable Progress ◽  
Made In

Recently, remarkable progress has been made in learning transferable representation across domains. Previous works in domain adaptation are majorly based on two techniques: domain-adversarial learning and self-training. However, domain-adversarial learning only aligns feature distributions between domains but does not consider whether the target features are discriminative. On the other hand, self-training utilizes the model predictions to enhance the discrimination of target features, but it is unable to explicitly align domain distributions. In order to combine the strengths of these two methods, we propose a novel method called Adversarial-Learned Loss for Domain Adaptation (ALDA). We first analyze the pseudo-label method, a typical self-training method. Nevertheless, there is a gap between pseudo-labels and the ground truth, which can cause incorrect training. Thus we introduce the confusion matrix, which is learned through an adversarial manner in ALDA, to reduce the gap and align the feature distributions. Finally, a new loss function is auto-constructed from the learned confusion matrix, which serves as the loss for unlabeled target samples. Our ALDA outperforms state-of-the-art approaches in four standard domain adaptation datasets. Our code is available at https://github.com/ZJULearning/ALDA.

scholarly journals Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss

2018 ◽  
Author(s):  
Qi Dou ◽  
Cheng Ouyang ◽  
Cheng Chen ◽  
Hao Chen ◽  
Pheng-Ann Heng
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Learning Approaches ◽  
Target Domain ◽  
Convolutional Network ◽  
Adversarial Learning ◽  
Biomedical Image ◽  
Convolutional Networks ◽  
Ct Data ◽  
Set Up

Convolutional networks (ConvNets) have achieved great successes in various challenging vision tasks. However, the performance of ConvNets would degrade when encountering the domain shift. The domain adaptation is more significant while challenging in the field of biomedical image analysis, where cross-modality data have largely different distributions. Given that annotating the medical data is especially expensive, the supervised transfer learning approaches are not quite optimal. In this paper, we propose an unsupervised domain adaptation framework with adversarial learning for cross-modality biomedical image segmentations. Specifically, our model is based on a dilated fully convolutional network for pixel-wise prediction. Moreover, we build a plug-and-play domain adaptation module (DAM) to map the target input to features which are aligned with source domain feature space. A domain critic module (DCM) is set up for discriminating the feature space of both domains. We optimize the DAM and DCM via an adversarial loss without using any target domain label. Our proposed method is validated by adapting a ConvNet trained with MRI images to unpaired CT data for cardiac structures segmentations, and achieved very promising results.

scholarly journals Consensus Adversarial Domain Adaptation

2019 ◽  
Vol 33 ◽  
pp. 5997-6004 ◽  
Author(s):  
Han Zou ◽  
Yuxun Zhou ◽  
Jianfei Yang ◽  
Huihan Liu ◽  
Hari Prasanna Das ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Spatial Dynamics ◽  
Feature Space ◽  
Target Domain ◽  
Adversarial Learning ◽  
Digit Recognition ◽  
Real World Problem ◽  
Benchmark Datasets ◽  
Free Gesture ◽  
Device Free

We propose a novel domain adaptation framework, namely Consensus Adversarial Domain Adaptation (CADA), that gives freedom to both target encoder and source encoder to embed data from both domains into a common domaininvariant feature space until they achieve consensus during adversarial learning. In this manner, the domain discrepancy can be further minimized in the embedded space, yielding more generalizable representations. The framework is also extended to establish a new few-shot domain adaptation scheme (F-CADA), that remarkably enhances the ADA performance by efficiently propagating a few labeled data once available in the target domain. Extensive experiments are conducted on the task of digit recognition across multiple benchmark datasets and a real-world problem involving WiFi-enabled device-free gesture recognition under spatial dynamics. The results show the compelling performance of CADA versus the state-of-the-art unsupervised domain adaptation (UDA) and supervised domain adaptation (SDA) methods. Numerical experiments also demonstrate that F-CADA can significantly improve the adaptation performance even with sparsely labeled data in the target domain.

scholarly journals DANE: Domain Adaptive Network Embedding

2019 ◽  
Author(s):  
Yizhou Zhang ◽  
Guojie Song ◽  
Lun Du ◽  
Shuwen Yang ◽  
Yilun Jin
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Structured Data ◽  
Network Embedding ◽  
Adaptive Network ◽  
Convolutional Network ◽  
Adversarial Learning ◽  
Multiple Networks ◽  
Shared Set ◽  
Machine Learning Models

Recent works reveal that network embedding techniques enable many machine learning models to handle diverse downstream tasks on graph structured data. However, as previous methods usually focus on learning embeddings for a single network, they can not learn representations transferable on multiple networks. Hence, it is important to design a network embedding algorithm that supports downstream model transferring on different networks, known as domain adaptation. In this paper, we propose a novel Domain Adaptive Network Embedding framework, which applies graph convolutional network to learn transferable embeddings. In DANE, nodes from multiple networks are encoded to vectors via a shared set of learnable parameters so that the vectors share an aligned embedding space. The distribution of embeddings on different networks are further aligned by adversarial learning regularization. In addition, DANE's advantage in learning transferable network embedding can be guaranteed theoretically. Extensive experiments reflect that the proposed framework outperforms other state-of-the-art network embedding baselines in cross-network domain adaptation tasks.

scholarly journals Zero Shot Learning via Low-rank Embedded Semantic AutoEncoder

2018 ◽  
Author(s):  
Yang Liu ◽  
Quanxue Gao ◽  
Jin Li ◽  
Jungong Han ◽  
Ling Shao
Keyword(s):  
State Of The Art ◽  
Feature Space ◽  
Original Data ◽  
Semantic Space ◽  
Dimensional Subspace ◽  
Low Rank ◽  
Visual Features ◽  
Visual Feature ◽  
Benchmark Datasets ◽  
Low Dimensional

Zero-shot learning (ZSL) has been widely researched and get successful in machine learning. Most existing ZSL methods aim to accurately recognize objects of unseen classes by learning a shared mapping from the feature space to a semantic space. However, such methods did not investigate in-depth whether the mapping can precisely reconstruct the original visual feature. Motivated by the fact that the data have low intrinsic dimensionality e.g. low-dimensional subspace. In this paper, we formulate a novel framework named Low-rank Embedded Semantic AutoEncoder (LESAE) to jointly seek a low-rank mapping to link visual features with their semantic representations. Taking the encoder-decoder paradigm, the encoder part aims to learn a low-rank mapping from the visual feature to the semantic space, while decoder part manages to reconstruct the original data with the learned mapping. In addition, a non-greedy iterative algorithm is adopted to solve our model. Extensive experiments on six benchmark datasets demonstrate its superiority over several state-of-the-art algorithms.

scholarly journals Ensemble-Based Out-of-Distribution Detection

Electronics ◽  
2021 ◽  
Vol 10 (5) ◽  
pp. 567
Author(s):  
Donghun Yang ◽  
Kien Mai Mai Ngoc ◽  
Iksoo Shin ◽  
Kyong-Ha Lee ◽  
Myunggwon Hwang
Keyword(s):  
Detection Method ◽  
State Of The Art ◽  
Metric Learning ◽  
Feature Space ◽  
Confidence Score ◽  
Distance Metric Learning ◽  
Current State ◽  
Overall Performance ◽  
Deep Learning Model

To design an efficient deep learning model that can be used in the real-world, it is important to detect out-of-distribution (OOD) data well. Various studies have been conducted to solve the OOD problem. The current state-of-the-art approach uses a confidence score based on the Mahalanobis distance in a feature space. Although it outperformed the previous approaches, the results were sensitive to the quality of the trained model and the dataset complexity. Herein, we propose a novel OOD detection method that can train more efficient feature space for OOD detection. The proposed method uses an ensemble of the features trained using the softmax-based classifier and the network based on distance metric learning (DML). Through the complementary interaction of these two networks, the trained feature space has a more clumped distribution and can fit well on the Gaussian distribution by class. Therefore, OOD data can be efficiently detected by setting a threshold in the trained feature space. To evaluate the proposed method, we applied our method to various combinations of image datasets. The results show that the overall performance of the proposed approach is superior to those of other methods, including the state-of-the-art approach, on any combination of datasets.

Sign in / Sign up

Export Citation Format

Share Document