scholarly journals Differentially Private Correlation Alignment for Domain Adaptation

2021 ◽  
Author(s):  
Kaizhong Jin ◽  
Xiang Cheng ◽  
Jiaxi Yang ◽  
Kaiyuan Shen
Keyword(s):  
Covariance Matrix ◽  
Domain Adaptation ◽  
Estimation Method ◽  
Training Data ◽  
Target Domain ◽  
Source Domain ◽  
Feature Spaces ◽  
Benchmark Datasets ◽  
Low Dimensional ◽  
Target Data

Domain adaptation solves a learning problem in a target domain by utilizing the training data in a different but related source domain. As a simple and efficient method for domain adaptation, correlation alignment transforms the distribution of the source domain by utilizing the covariance matrix of the target domain, such that a model trained on the transformed source data can be applied to the target data. However, when source and target domains come from different institutes, exchanging information between the two domains might pose a potential privacy risk. In this paper, for the first time, we propose a differentially private correlation alignment approach for domain adaptation called PRIMA, which can provide privacy guarantees for both the source and target data. In PRIMA, to relieve the performance degradation caused by perturbing the covariance matrix in high dimensional setting, we present a random subspace ensemble based covariance estimation method which splits the feature spaces of source and target data into several low dimensional subspaces. Moreover, since perturbing the covariance matrix may destroy its positive semi-definiteness, we develop a shrinking based method for the recovery of positive semi-definiteness of the covariance matrix. Experimental results on standard benchmark datasets confirm the effectiveness of our approach.

scholarly journals Learning Discriminative Correlation Subspace for Heterogeneous Domain Adaptation

2017 ◽  
Author(s):  
Yuguang Yan ◽  
Wen Li ◽  
Michael Ng ◽  
Mingkui Tan ◽  
Hanrui Wu ◽  
...  
Keyword(s):  
Optimization Problem ◽  
Domain Adaptation ◽  
Data Sets ◽  
Target Domain ◽  
Real World Data ◽  
Discriminative Ability ◽  
Convex Optimization Problem ◽  
Alternating Direction ◽  
Feature Spaces ◽  
Target Data

Domain adaptation aims to reduce the effort on collecting and annotating target data by leveraging knowledge from a different source domain. The domain adaptation problem will become extremely challenging when the feature spaces of the source and target domains are different, which is also known as the heterogeneous domain adaptation (HDA) problem. In this paper, we propose a novel HDA method to find the optimal discriminative correlation subspace for the source and target data. The discriminative correlation subspace is inherited from the canonical correlation subspace between the source and target data, and is further optimized to maximize the discriminative ability for the target domain classifier. We formulate a joint objective in order to simultaneously learn the discriminative correlation subspace and the target domain classifier. We then apply an alternating direction method of multiplier (ADMM) algorithm to address the resulting non-convex optimization problem. Comprehensive experiments on two real-world data sets demonstrate the effectiveness of the proposed method compared to the state-of-the-art methods.

scholarly journals Distributional Correspondence Indexing for Cross-Lingual and Cross-Domain Sentiment Classification (Extended Abstract)

2018 ◽  
Author(s):  
Alejandro Moreo Fernández ◽  
Andrea Esuli ◽  
Fabrizio Sebastiani
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Training Data ◽  
Target Domain ◽  
Source Domain ◽  
Machine Learning Methods ◽  
Cross Domain ◽  
Current State ◽  
Cross Lingual

Domain Adaptation (DA) techniques aim at enabling machine learning methods learn effective classifiers for a “target” domain when the only available training data belongs to a different “source” domain. In this extended abstract, we briefly describe our new DA method called Distributional Correspondence Indexing (DCI) for sentiment classification. DCI derives term representations in a vector space common to both domains where each dimension reflects its distributional correspondence to a pivot, i.e., to a highly predictive term that behaves similarly across domains. The experiments we have conducted show that DCI obtains better performance than current state-of-the-art techniques for cross-lingual and cross-domain sentiment classification.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

scholarly journals Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation

2019 ◽  
Vol 33 ◽  
pp. 5581-5588 ◽  
Author(s):  
Yonghao Xu ◽  
Bo Du ◽  
Lefei Zhang ◽  
Qian Zhang ◽  
Guoli Wang ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Great Success ◽  
Learning Models ◽  
Target Domain ◽  
Attention Networks ◽  
Source Domain ◽  
Benchmark Datasets ◽  
Different Levels

Recent years have witnessed the great success of deep learning models in semantic segmentation. Nevertheless, these models may not generalize well to unseen image domains due to the phenomenon of domain shift. Since pixel-level annotations are laborious to collect, developing algorithms which can adapt labeled data from source domain to target domain is of great significance. To this end, we propose self-ensembling attention networks to reduce the domain gap between different datasets. To the best of our knowledge, the proposed method is the first attempt to introduce selfensembling model to domain adaptation for semantic segmentation, which provides a different view on how to learn domain-invariant features. Besides, since different regions in the image usually correspond to different levels of domain gap, we introduce the attention mechanism into the proposed framework to generate attention-aware features, which are further utilized to guide the calculation of consistency loss in the target domain. Experiments on two benchmark datasets demonstrate that the proposed framework can yield competitive performance compared with the state of the art methods.

scholarly journals Multi-Source Domain Adaptation for Visual Sentiment Classification

2020 ◽  
Vol 34 (03) ◽  
pp. 2661-2668
Author(s):  
Chuang Lin ◽  
Sicheng Zhao ◽  
Lei Meng ◽  
Tat-Seng Chua
Keyword(s):  
Domain Adaptation ◽  
Sentiment Classification ◽  
Similar Distribution ◽  
Single Source ◽  
Target Domain ◽  
Generative Adversarial Network ◽  
Source Domain ◽  
Adversarial Network ◽  
Latent Space ◽  
Benchmark Datasets

Existing domain adaptation methods on visual sentiment classification typically are investigated under the single-source scenario, where the knowledge learned from a source domain of sufficient labeled data is transferred to the target domain of loosely labeled or unlabeled data. However, in practice, data from a single source domain usually have a limited volume and can hardly cover the characteristics of the target domain. In this paper, we propose a novel multi-source domain adaptation (MDA) method, termed Multi-source Sentiment Generative Adversarial Network (MSGAN), for visual sentiment classification. To handle data from multiple source domains, it learns to find a unified sentiment latent space where data from both the source and target domains share a similar distribution. This is achieved via cycle consistent adversarial learning in an end-to-end manner. Extensive experiments conducted on four benchmark datasets demonstrate that MSGAN significantly outperforms the state-of-the-art MDA approaches for visual sentiment classification.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

scholarly journals Synthetic Source Universal Domain Adaptation through Contrastive Learning

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7539
Author(s):  
Jungchan Cho
Keyword(s):  
Domain Adaptation ◽  
Synthetic Data ◽  
Real Data ◽  
Target Domain ◽  
Source Domain ◽  
Universal Domain ◽  
Model Training ◽  
Using Data ◽  
Imaging Sensors ◽  
Target Data

Universal domain adaptation (UDA) is a crucial research topic for efficient deep learning model training using data from various imaging sensors. However, its development is affected by unlabeled target data. Moreover, the nonexistence of prior knowledge of the source and target domain makes it more challenging for UDA to train models. I hypothesize that the degradation of trained models in the target domain is caused by the lack of direct training loss to improve the discriminative power of the target domain data. As a result, the target data adapted to the source representations is biased toward the source domain. I found that the degradation was more pronounced when I used synthetic data for the source domain and real data for the target domain. In this paper, I propose a UDA method with target domain contrastive learning. The proposed method enables models to leverage synthetic data for the source domain and train the discriminativeness of target features in an unsupervised manner. In addition, the target domain feature extraction network is shared with the source domain classification task, preventing unnecessary computational growth. Extensive experimental results on VisDa-2017 and MNIST to SVHN demonstrated that the proposed method significantly outperforms the baseline by 2.7% and 5.1%, respectively.

scholarly journals Domain Adaptation for Semantic Parsing

2020 ◽  
Author(s):  
Zechang Li ◽  
Yuxuan Lai ◽  
Yansong Feng ◽  
Dongyan Zhao
Keyword(s):  
Domain Adaptation ◽  
Specific Information ◽  
Semantic Parsing ◽  
Target Domain ◽  
Source Domain ◽  
Domain Specific ◽  
The Difference ◽  
Two Stages ◽  
Coarse To Fine ◽  
Target Data

Recently, semantic parsing has attracted much attention in the community. Although many neural modeling efforts have greatly improved the performance, it still suffers from the data scarcity issue. In this paper, we propose a novel semantic parser for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic parser benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages, i.e., focusing on domain invariant and domain specific information, respectively. In the coarse stage, our novel domain discrimination component and domain relevance attention encourage the model to learn transferable domain general structures. In the fine stage, the model is guided to concentrate on domain related details. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies. Additionally, we show that our model can well exploit limited target data to capture the difference between the source and target domain, even when the target domain has far fewer training instances.

scholarly journals Unsupervised multi-source domain adaptation with no observable source data

PLoS ONE ◽  
2021 ◽  
Vol 16 (7) ◽  
pp. e0253415
Author(s):  
Hyunsik Jeon ◽  
Seongmin Lee ◽  
U Kang
Keyword(s):  
Real World ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Multiple Source ◽  
Multiple Sources ◽  
Target Domain ◽  
Source Domain ◽  
Source Data ◽  
Real World Datasets ◽  
Target Data

Given trained models from multiple source domains, how can we predict the labels of unlabeled data in a target domain? Unsupervised multi-source domain adaptation (UMDA) aims for predicting the labels of unlabeled target data by transferring the knowledge of multiple source domains. UMDA is a crucial problem in many real-world scenarios where no labeled target data are available. Previous approaches in UMDA assume that data are observable over all domains. However, source data are not easily accessible due to privacy or confidentiality issues in a lot of practical scenarios, although classifiers learned in source domains are readily available. In this work, we target data-free UMDA where source data are not observable at all, a novel problem that has not been studied before despite being very realistic and crucial. To solve data-free UMDA, we propose DEMS (Data-free Exploitation of Multiple Sources), a novel architecture that adapts target data to source domains without exploiting any source data, and estimates the target labels by exploiting pre-trained source classifiers. Extensive experiments for data-free UMDA on real-world datasets show that DEMS provides the state-of-the-art accuracy which is up to 27.5% point higher than that of the best baseline.

scholarly journals Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation

2021 ◽  
Author(s):  
Zhen Qiu ◽  
Yifan Zhang ◽  
Hongbin Lin ◽  
Shuaicheng Niu ◽  
Yanxia Liu ◽  
...  
Keyword(s):  
Data Privacy ◽  
Domain Adaptation ◽  
Adaptation Strategy ◽  
Source Model ◽  
Target Domain ◽  
Benchmark Datasets ◽  
Classification Boundary ◽  
Hidden Knowledge ◽  
Two Stages ◽  
Target Data

We study a practical domain adaptation task, called source-free unsupervised domain adaptation (UDA) problem, in which we cannot access source domain data due to data privacy issues but only a pre-trained source model and unlabeled target data are available. This task, however, is very difficult due to one key challenge: the lack of source data and target domain labels makes model adaptation very challenging. To address this, we propose to mine the hidden knowledge in the source model and exploit it to generate source avatar prototypes (i.e. representative features for each source class) as well as target pseudo labels for domain alignment. To this end, we propose a Contrastive Prototype Generation and Adaptation (CPGA) method. Specifically, CPGA consists of two stages: (1) prototype generation: by exploring the classification boundary information of the source model, we train a prototype generator to generate avatar prototypes via contrastive learning. (2) prototype adaptation: based on the generated source prototypes and target pseudo labels, we develop a new robust contrastive prototype adaptation strategy to align each pseudo-labeled target data to the corresponding source prototypes. Extensive experiments on three UDA benchmark datasets demonstrate the effectiveness and superiority of the proposed method.

Sign in / Sign up

Export Citation Format

Share Document