scholarly journals Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation

2020 ◽  
Author(s):  
Yiyang Zhang ◽  
Feng Liu ◽  
Zhen Fang ◽  
Bo Yuan ◽  
Guangquan Zhang ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Unlabeled Data ◽  
Target Domain ◽  
Source Domain ◽  
Label Data ◽  
Unsupervised Domain Adaptation ◽  
Adversarial Network ◽  
True Label ◽  
Source Data ◽  
One Step

In unsupervised domain adaptation (UDA), classifiers for the target domain are trained with massive true-label data from the source domain and unlabeled data from the target domain. However, it may be difficult to collect fully-true-label data in a source domain given limited budget. To mitigate this problem, we consider a novel problem setting where the classifier for the target domain has to be trained with complementary-label data from the source domain and unlabeled data from the target domain named budget-friendly UDA (BFUDA). The key benefit is that it is much less costly to collect complementary-label source data (required by BFUDA) than collecting the true-label source data (required by ordinary UDA). To this end, complementary label adversarial network (CLARINET) is proposed to solve the BFUDA problem. CLARINET maintains two deep networks simultaneously, where one focuses on classifying complementary-label source data and the other takes care of the source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines.

Correlation alignment with attention mechanism for unsupervised domain adaptation

2021 ◽  
pp. 1-7
Author(s):  
Rong Chen ◽  
Chongguang Ren
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Transfer Process ◽  
Negative Transfer ◽  
Domain Adaptation ◽  
Attention Mechanism ◽  
Target Domain ◽  
Source Domain ◽  
Second Order Statistics ◽  
Unsupervised Domain Adaptation

Domain adaptation aims to solve the problems of lacking labels. Most existing works of domain adaptation mainly focus on aligning the feature distributions between the source and target domain. However, in the field of Natural Language Processing, some of the words in different domains convey different sentiment. Thus not all features of the source domain should be transferred, and it would cause negative transfer when aligning the untransferable features. To address this issue, we propose a Correlation Alignment with Attention mechanism for unsupervised Domain Adaptation (CAADA) model. In the model, an attention mechanism is introduced into the transfer process for domain adaptation, which can capture the positively transferable features in source and target domain. Moreover, the CORrelation ALignment (CORAL) loss is utilized to minimize the domain discrepancy by aligning the second-order statistics of the positively transferable features extracted by the attention mechanism. Extensive experiments on the Amazon review dataset demonstrate the effectiveness of CAADA method.

scholarly journals Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

2021 ◽  
Author(s):  
Jiahao Fan ◽  
Hangyu Zhu ◽  
Xinyu Jiang ◽  
Long Meng ◽  
Cong Fu ◽  
...  
Keyword(s):  
Large Scale ◽  
Domain Adaptation ◽  
Source Model ◽  
Deep Sleep ◽  
Sleep Staging ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Generalization Problem ◽  
Source Models

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.<br>

scholarly journals Multi-Source Domain Adaptation for Visual Sentiment Classification

2020 ◽  
Vol 34 (03) ◽  
pp. 2661-2668
Author(s):  
Chuang Lin ◽  
Sicheng Zhao ◽  
Lei Meng ◽  
Tat-Seng Chua
Keyword(s):  
Domain Adaptation ◽  
Sentiment Classification ◽  
Similar Distribution ◽  
Single Source ◽  
Target Domain ◽  
Generative Adversarial Network ◽  
Source Domain ◽  
Adversarial Network ◽  
Latent Space ◽  
Benchmark Datasets

Existing domain adaptation methods on visual sentiment classification typically are investigated under the single-source scenario, where the knowledge learned from a source domain of sufficient labeled data is transferred to the target domain of loosely labeled or unlabeled data. However, in practice, data from a single source domain usually have a limited volume and can hardly cover the characteristics of the target domain. In this paper, we propose a novel multi-source domain adaptation (MDA) method, termed Multi-source Sentiment Generative Adversarial Network (MSGAN), for visual sentiment classification. To handle data from multiple source domains, it learns to find a unified sentiment latent space where data from both the source and target domains share a similar distribution. This is achieved via cycle consistent adversarial learning in an end-to-end manner. Extensive experiments conducted on four benchmark datasets demonstrate that MSGAN significantly outperforms the state-of-the-art MDA approaches for visual sentiment classification.

scholarly journals Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits

2020 ◽  
Vol 34 (05) ◽  
pp. 7830-7838 ◽  
Author(s):  
Han Guo ◽  
Ramakanth Pasunuru ◽  
Mohit Bansal
Keyword(s):  
Loss Function ◽  
Optimal Trajectory ◽  
Domain Adaptation ◽  
Learning Algorithm ◽  
Data Distribution ◽  
Distance Measures ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Additional Loss

Domain adaptation performance of a learning algorithm on a target domain is a function of its source domain error and a divergence measure between the data distribution of these two domains. We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between domains based on sample estimates. We first conduct analysis experiments to show which of these distance measures can best differentiate samples from same versus different domains, and are correlated with empirical results. Next, we develop a DistanceNet model which uses these distance measures, or a mixture of these distance measures, as an additional loss function to be minimized jointly with the task's loss function, so as to achieve better unsupervised domain adaptation. Finally, we extend this model to a novel DistanceNet-Bandit model, which employs a multi-armed bandit controller to dynamically switch between multiple source domains and allow the model to learn an optimal trajectory and mixture of domains for transfer to the low-resource target domain. We conduct experiments on popular sentiment analysis datasets with several diverse domains and show that our DistanceNet model, as well as its dynamic bandit variant, can outperform competitive baselines in the context of unsupervised domain adaptation.

scholarly journals Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling

2020 ◽  
Vol 34 (04) ◽  
pp. 6243-6250 ◽  
Author(s):  
Qian Wang ◽  
Toby Breckon
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Structured Prediction ◽  
Target Domain ◽  
Source Domain ◽  
Domain Specific ◽  
Unsupervised Domain Adaptation ◽  
Deep Feature ◽  
Significant Performance ◽  
Error Accumulation

Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain suffer from significant performance drop when directly applied to the samples from the target domain. To address this issue, different approaches have been proposed to learn domain-invariant features or domain-specific classifiers. In either case, the lack of labeled samples in the target domain can be an issue which is usually overcome by pseudo-labeling. Inaccurate pseudo-labeling, however, could result in catastrophic error accumulation during learning. In this paper, we propose a novel selective pseudo-labeling strategy based on structured prediction. The idea of structured prediction is inspired by the fact that samples in the target domain are well clustered within the deep feature space so that unsupervised clustering analysis can be used to facilitate accurate pseudo-labeling. Experimental results on four datasets (i.e. Office-Caltech, Office31, ImageCLEF-DA and Office-Home) validate our approach outperforms contemporary state-of-the-art methods.

scholarly journals Unsupervised Domain Adaptation Method Based on Discriminant Sample Selection

2020 ◽  
Vol 38 (4) ◽  
pp. 828-837
Author(s):  
Linlin Wu ◽  
Guohua Peng ◽  
Weidong Yan
Keyword(s):  
Density Estimation ◽  
Classification Accuracy ◽  
Domain Adaptation ◽  
Sample Selection ◽  
Previous Method ◽  
Probability Density Estimation ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Adaptation Method

In order to solve the problem that low classification accuracy caused by the different distribution of training set and test set, an unsupervised domain adaptation method based on discriminant sample selection (DSS) is proposed. DSS projects the samples of different domains onto a same subspace to reduce the distribution discrepancy between the source domain and the target domain, and weights the source domain instances to make the samples more discriminant. Different from the previous method based on the probability density estimation of samples, DSS tries to obtain the sample weights by solving a quadratic programming problem, which avoids the distribution estimation of samples and can be applied to any fields without suffering from the dimensional trouble caused by high-dimensional density estimation. Finally, DSS congregates the same classes by minimizing the intra-class distance. Experimental results show that the proposed method improves the classification accuracy and robustness.

scholarly journals Unsupervised multi-source domain adaptation with no observable source data

PLoS ONE ◽  
2021 ◽  
Vol 16 (7) ◽  
pp. e0253415
Author(s):  
Hyunsik Jeon ◽  
Seongmin Lee ◽  
U Kang
Keyword(s):  
Real World ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Multiple Source ◽  
Multiple Sources ◽  
Target Domain ◽  
Source Domain ◽  
Source Data ◽  
Real World Datasets ◽  
Target Data

Given trained models from multiple source domains, how can we predict the labels of unlabeled data in a target domain? Unsupervised multi-source domain adaptation (UMDA) aims for predicting the labels of unlabeled target data by transferring the knowledge of multiple source domains. UMDA is a crucial problem in many real-world scenarios where no labeled target data are available. Previous approaches in UMDA assume that data are observable over all domains. However, source data are not easily accessible due to privacy or confidentiality issues in a lot of practical scenarios, although classifiers learned in source domains are readily available. In this work, we target data-free UMDA where source data are not observable at all, a novel problem that has not been studied before despite being very realistic and crucial. To solve data-free UMDA, we propose DEMS (Data-free Exploitation of Multiple Sources), a novel architecture that adapts target data to source domains without exploiting any source data, and estimates the target labels by exploiting pre-trained source classifiers. Extensive experiments for data-free UMDA on real-world datasets show that DEMS provides the state-of-the-art accuracy which is up to 27.5% point higher than that of the best baseline.

scholarly journals Category-Sensitive Domain Adaptation for Land Cover Mapping in Aerial Scenes

2019 ◽  
Vol 11 (22) ◽  
pp. 2631 ◽  
Author(s):  
Bo Fang ◽  
Rong Kou ◽  
Li Pan ◽  
Pengfei Chen
Keyword(s):  
Land Cover ◽  
Domain Adaptation ◽  
Feature Space ◽  
Aerial Images ◽  
Land Cover Mapping ◽  
Target Domain ◽  
Generative Adversarial Network ◽  
Source Domain ◽  
Adversarial Network ◽  
Semantic Labeling

Since manually labeling aerial images for pixel-level classification is expensive and time-consuming, developing strategies for land cover mapping without reference labels is essential and meaningful. As an efficient solution for this issue, domain adaptation has been widely utilized in numerous semantic labeling-based applications. However, current approaches generally pursue the marginal distribution alignment between the source and target features and ignore the category-level alignment. Therefore, directly applying them to land cover mapping leads to unsatisfactory performance in the target domain. In our research, to address this problem, we embed a geometry-consistent generative adversarial network (GcGAN) into a co-training adversarial learning network (CtALN), and then develop a category-sensitive domain adaptation (CsDA) method for land cover mapping using very-high-resolution (VHR) optical aerial images. The GcGAN aims to eliminate the domain discrepancies between labeled and unlabeled images while retaining their intrinsic land cover information by translating the features of the labeled images from the source domain to the target domain. Meanwhile, the CtALN aims to learn a semantic labeling model in the target domain with the translated features and corresponding reference labels. By training this hybrid framework, our method learns to distill knowledge from the source domain and transfers it to the target domain, while preserving not only global domain consistency, but also category-level consistency between labeled and unlabeled images in the feature space. The experimental results between two airborne benchmark datasets and the comparison with other state-of-the-art methods verify the robustness and superiority of our proposed CsDA.

scholarly journals Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

2021 ◽  
Author(s):  
Jiahao Fan ◽  
Hangyu Zhu ◽  
Xinyu Jiang ◽  
Long Meng ◽  
Cong Fu ◽  
...  
Keyword(s):  
Large Scale ◽  
Domain Adaptation ◽  
Source Model ◽  
Deep Sleep ◽  
Sleep Staging ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Generalization Problem ◽  
Source Models

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.<br>

scholarly journals Unsupervised Domain Expansion for Visual Categorization

2021 ◽  
Vol 17 (4) ◽  
pp. 1-24
Author(s):  
Jie Wang ◽  
Kaibin Tian ◽  
Dayong Ding ◽  
Gang Yang ◽  
Xirong Li
Keyword(s):  
Domain Adaptation ◽  
Unlabeled Data ◽  
Target Domain ◽  
Source Domain ◽  
Visual Categorization ◽  
Performance Loss ◽  
Deep Model ◽  
Knowledge Distillation ◽  
The Cost

Expanding visual categorization into a novel domain without the need of extra annotation has been a long-term interest for multimedia intelligence. Previously, this challenge has been approached by unsupervised domain adaptation (UDA). Given labeled data from a source domain and unlabeled data from a target domain, UDA seeks for a deep representation that is both discriminative and domain-invariant. While UDA focuses on the target domain, we argue that the performance on both source and target domains matters, as in practice which domain a test example comes from is unknown. In this article, we extend UDA by proposing a new task called unsupervised domain expansion (UDE), which aims to adapt a deep model for the target domain with its unlabeled data, meanwhile maintaining the model’s performance on the source domain. We propose Knowledge Distillation Domain Expansion (KDDE) as a general method for the UDE task. Its domain-adaptation module can be instantiated with any existing model. We develop a knowledge distillation-based learning mechanism, enabling KDDE to optimize a single objective wherein the source and target domains are equally treated. Extensive experiments on two major benchmarks, i.e., Office-Home and DomainNet, show that KDDE compares favorably against four competitive baselines, i.e., DDC, DANN, DAAN, and CDAN, for both UDA and UDE tasks. Our study also reveals that the current UDA models improve their performance on the target domain at the cost of noticeable performance loss on the source domain.

Sign in / Sign up

Export Citation Format

Share Document