scholarly journals MS-DIAL: Multi-Source Domain Alignment Layers for Unsupervised Domain Adaptation

2020 ◽  
Author(s):  
Lucas Fernando Alvarenga e Silva ◽  
Jurandy Almeida
Keyword(s):  
Deep Neural Networks ◽  
Domain Adaptation ◽  
Research Area ◽  
Multiple Sources ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Public Datasets ◽  
Alignment Layers ◽  
Domain Alignment ◽  
Different Sources

In general, deep neural networks trained on a given labeled dataset are expected to produce equivalent results when tested on a new unlabeled dataset. However, data are generally collected by different devices or under varying conditions and thus they often are not part of a same domain, yielding poor results. This is due to the domain shift between data distributions and has been the goal of a research area known as unsupervised domain adaptation. Many prior works have been designed to transfer knowledge between two domains: one source to one target. Since data may be taken from different sources and with different distributions, multi-source domain adaptation has received increasing attention. This paper presents the Multi-Source DomaIn Alignment Layers (MS-DIAL), which reduce the domain shift between multiple sources and a given target by embedding domain alignment layers in any given network. Except for the embedded layers, all the other network parameters are shared among all domains, saving processing time and memory usage. Experiments were performed on digit and object recognition tasks with five public datasets widely used to evaluate domain adaptation methods. Results show that the proposed method is promising and outperforms state-of-the-art approaches.

scholarly journals Just DIAL: DomaIn Alignment Layers for Unsupervised Domain Adaptation

2017 ◽  
pp. 357-369 ◽  
Author(s):  
Fabio Maria Carlucci ◽  
Lorenzo Porzi ◽  
Barbara Caputo ◽  
Elisa Ricci ◽  
Samuel Rota Bulò
Keyword(s):  
Domain Adaptation ◽  
Unsupervised Domain Adaptation ◽  
Alignment Layers ◽  
Domain Alignment

Correlation alignment with attention mechanism for unsupervised domain adaptation

2021 ◽  
pp. 1-7
Author(s):  
Rong Chen ◽  
Chongguang Ren
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Transfer Process ◽  
Negative Transfer ◽  
Domain Adaptation ◽  
Attention Mechanism ◽  
Target Domain ◽  
Source Domain ◽  
Second Order Statistics ◽  
Unsupervised Domain Adaptation

Domain adaptation aims to solve the problems of lacking labels. Most existing works of domain adaptation mainly focus on aligning the feature distributions between the source and target domain. However, in the field of Natural Language Processing, some of the words in different domains convey different sentiment. Thus not all features of the source domain should be transferred, and it would cause negative transfer when aligning the untransferable features. To address this issue, we propose a Correlation Alignment with Attention mechanism for unsupervised Domain Adaptation (CAADA) model. In the model, an attention mechanism is introduced into the transfer process for domain adaptation, which can capture the positively transferable features in source and target domain. Moreover, the CORrelation ALignment (CORAL) loss is utilized to minimize the domain discrepancy by aligning the second-order statistics of the positively transferable features extracted by the attention mechanism. Extensive experiments on the Amazon review dataset demonstrate the effectiveness of CAADA method.

scholarly journals Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

2021 ◽  
Author(s):  
Jiahao Fan ◽  
Hangyu Zhu ◽  
Xinyu Jiang ◽  
Long Meng ◽  
Cong Fu ◽  
...  
Keyword(s):  
Large Scale ◽  
Domain Adaptation ◽  
Source Model ◽  
Deep Sleep ◽  
Sleep Staging ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Generalization Problem ◽  
Source Models

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.<br>

scholarly journals Opinion mining using combinational approach for different domains

2019 ◽  
Vol 9 (4) ◽  
pp. 3307
Author(s):  
Jyoti Sandesh Deshmukh ◽  
Amiya Kumar Tripathy ◽  
Dilendra Hiran
Keyword(s):  
Maximum Entropy ◽  
Cross Validation ◽  
Opinion Mining ◽  
Domain Adaptation ◽  
Online Reviews ◽  
Research Area ◽  
Source Domain ◽  
Different Types ◽  
Fold Cross Validation ◽  
Target Data

An increase in use of web produces large content of information about products. Online reviews are used to make decision by peoples. Opinion mining is vast research area in which different types of reviews are analyzed. Several issues are existing in this area. Domain adaptation is emerging issue in opinion mining. Labling of data for every domain is time consuming and costly task. Hence the need arises for model that train one domain and applied it on other domain reducing cost aswell as time. This is called domain adaptation which is addressed in this paper. Using maximum entropy and clustering technique source domains data is trained. Trained data from source domain is applied on target data to labeling purpose A result shows moderate accuracy for 5 fold cross validation and combination of source domains for Blitzer et al (2007) multi domain product dataset.

scholarly journals Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits

2020 ◽  
Vol 34 (05) ◽  
pp. 7830-7838 ◽  
Author(s):  
Han Guo ◽  
Ramakanth Pasunuru ◽  
Mohit Bansal
Keyword(s):  
Loss Function ◽  
Optimal Trajectory ◽  
Domain Adaptation ◽  
Learning Algorithm ◽  
Data Distribution ◽  
Distance Measures ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Additional Loss

Domain adaptation performance of a learning algorithm on a target domain is a function of its source domain error and a divergence measure between the data distribution of these two domains. We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between domains based on sample estimates. We first conduct analysis experiments to show which of these distance measures can best differentiate samples from same versus different domains, and are correlated with empirical results. Next, we develop a DistanceNet model which uses these distance measures, or a mixture of these distance measures, as an additional loss function to be minimized jointly with the task's loss function, so as to achieve better unsupervised domain adaptation. Finally, we extend this model to a novel DistanceNet-Bandit model, which employs a multi-armed bandit controller to dynamically switch between multiple source domains and allow the model to learn an optimal trajectory and mixture of domains for transfer to the low-resource target domain. We conduct experiments on popular sentiment analysis datasets with several diverse domains and show that our DistanceNet model, as well as its dynamic bandit variant, can outperform competitive baselines in the context of unsupervised domain adaptation.

scholarly journals Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling

2020 ◽  
Vol 34 (04) ◽  
pp. 6243-6250 ◽  
Author(s):  
Qian Wang ◽  
Toby Breckon
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Structured Prediction ◽  
Target Domain ◽  
Source Domain ◽  
Domain Specific ◽  
Unsupervised Domain Adaptation ◽  
Deep Feature ◽  
Significant Performance ◽  
Error Accumulation

Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain suffer from significant performance drop when directly applied to the samples from the target domain. To address this issue, different approaches have been proposed to learn domain-invariant features or domain-specific classifiers. In either case, the lack of labeled samples in the target domain can be an issue which is usually overcome by pseudo-labeling. Inaccurate pseudo-labeling, however, could result in catastrophic error accumulation during learning. In this paper, we propose a novel selective pseudo-labeling strategy based on structured prediction. The idea of structured prediction is inspired by the fact that samples in the target domain are well clustered within the deep feature space so that unsupervised clustering analysis can be used to facilitate accurate pseudo-labeling. Experimental results on four datasets (i.e. Office-Caltech, Office31, ImageCLEF-DA and Office-Home) validate our approach outperforms contemporary state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document