Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.<br>

Download Full-text

Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

10.36227/techrxiv.17212184 ◽

2021 ◽

Author(s):

Jiahao Fan ◽

Hangyu Zhu ◽

Xinyu Jiang ◽

Long Meng ◽

Cong Fu ◽

...

Keyword(s):

Large Scale ◽

Domain Adaptation ◽

Source Model ◽

Deep Sleep ◽

Sleep Staging ◽

Target Domain ◽

Source Domain ◽

Unsupervised Domain Adaptation ◽

Generalization Problem ◽

Source Models

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.<br>

Download Full-text

Correlation alignment with attention mechanism for unsupervised domain adaptation

Web Intelligence ◽

10.3233/web-210447 ◽

2021 ◽

pp. 1-7

Author(s):

Rong Chen ◽

Chongguang Ren

Keyword(s):

Natural Language Processing ◽

Language Processing ◽

Transfer Process ◽

Negative Transfer ◽

Domain Adaptation ◽

Attention Mechanism ◽

Target Domain ◽

Source Domain ◽

Second Order Statistics ◽

Unsupervised Domain Adaptation

Domain adaptation aims to solve the problems of lacking labels. Most existing works of domain adaptation mainly focus on aligning the feature distributions between the source and target domain. However, in the field of Natural Language Processing, some of the words in different domains convey different sentiment. Thus not all features of the source domain should be transferred, and it would cause negative transfer when aligning the untransferable features. To address this issue, we propose a Correlation Alignment with Attention mechanism for unsupervised Domain Adaptation (CAADA) model. In the model, an attention mechanism is introduced into the transfer process for domain adaptation, which can capture the positively transferable features in source and target domain. Moreover, the CORrelation ALignment (CORAL) loss is utilized to minimize the domain discrepancy by aligning the second-order statistics of the positively transferable features extracted by the attention mechanism. Extensive experiments on the Amazon review dataset demonstrate the effectiveness of CAADA method.

Download Full-text

FeatureTransfer: Unsupervised Domain Adaptation for Cross-Domain Deepfake Detection

Security and Communication Networks ◽

10.1155/2021/9942754 ◽

2021 ◽

Vol 2021 ◽

pp. 1-8

Author(s):

Baoying Chen ◽

Shunquan Tan

Keyword(s):

Large Scale ◽

Detection Method ◽

Domain Adaptation ◽

Third Party ◽

Detection Methods ◽

Target Domain ◽

Feature Vectors ◽

Unsupervised Domain Adaptation ◽

Cross Domain ◽

Overfitting Problem

Recently, various Deepfake detection methods have been proposed, and most of them are based on convolutional neural networks (CNNs). These detection methods suffer from overfitting on the source dataset and do not perform well on cross-domain datasets which have different distributions from the source dataset. To address these limitations, a new method named FeatureTransfer is proposed in this paper, which is a two-stage Deepfake detection method combining with transfer learning. Firstly, The CNN model pretrained on a third-party large-scale Deepfake dataset can be used to extract the more transferable feature vectors of Deepfake videos in the source and target domains. Secondly, these feature vectors are fed into the domain-adversarial neural network based on backpropagation (BP-DANN) for unsupervised domain adaptive training, where the videos in the source domain have real or fake labels, while the videos in the target domain are unlabelled. The experimental results indicate that the proposed method FeatureTransfer can effectively solve the overfitting problem in Deepfake detection and greatly improve the performance of cross-dataset evaluation.

Download Full-text

Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6288 ◽

2020 ◽

Vol 34 (05) ◽

pp. 7830-7838 ◽

Cited By ~ 1

Author(s):

Han Guo ◽

Ramakanth Pasunuru ◽

Mohit Bansal

Keyword(s):

Loss Function ◽

Optimal Trajectory ◽

Domain Adaptation ◽

Learning Algorithm ◽

Data Distribution ◽

Distance Measures ◽

Target Domain ◽

Source Domain ◽

Unsupervised Domain Adaptation ◽

Additional Loss

Domain adaptation performance of a learning algorithm on a target domain is a function of its source domain error and a divergence measure between the data distribution of these two domains. We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between domains based on sample estimates. We first conduct analysis experiments to show which of these distance measures can best differentiate samples from same versus different domains, and are correlated with empirical results. Next, we develop a DistanceNet model which uses these distance measures, or a mixture of these distance measures, as an additional loss function to be minimized jointly with the task's loss function, so as to achieve better unsupervised domain adaptation. Finally, we extend this model to a novel DistanceNet-Bandit model, which employs a multi-armed bandit controller to dynamically switch between multiple source domains and allow the model to learn an optimal trajectory and mixture of domains for transfer to the low-resource target domain. We conduct experiments on popular sentiment analysis datasets with several diverse domains and show that our DistanceNet model, as well as its dynamic bandit variant, can outperform competitive baselines in the context of unsupervised domain adaptation.

Download Full-text

Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.6091 ◽

2020 ◽

Vol 34 (04) ◽

pp. 6243-6250 ◽

Cited By ~ 2

Author(s):

Qian Wang ◽

Toby Breckon

Keyword(s):

Domain Adaptation ◽

Feature Space ◽

Structured Prediction ◽

Target Domain ◽

Source Domain ◽

Domain Specific ◽

Unsupervised Domain Adaptation ◽

Deep Feature ◽

Significant Performance ◽

Error Accumulation

Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain suffer from significant performance drop when directly applied to the samples from the target domain. To address this issue, different approaches have been proposed to learn domain-invariant features or domain-specific classifiers. In either case, the lack of labeled samples in the target domain can be an issue which is usually overcome by pseudo-labeling. Inaccurate pseudo-labeling, however, could result in catastrophic error accumulation during learning. In this paper, we propose a novel selective pseudo-labeling strategy based on structured prediction. The idea of structured prediction is inspired by the fact that samples in the target domain are well clustered within the deep feature space so that unsupervised clustering analysis can be used to facilitate accurate pseudo-labeling. Experimental results on four datasets (i.e. Office-Caltech, Office31, ImageCLEF-DA and Office-Home) validate our approach outperforms contemporary state-of-the-art methods.

Download Full-text

Few shot domain adaptation for in situ macromolecule structural classification in cryoelectron tomograms

Bioinformatics ◽

10.1093/bioinformatics/btaa671 ◽

2020 ◽

Author(s):

Liangyong Yu ◽

Ran Li ◽

Xiangrui Zeng ◽

Hongyi Wang ◽

Jie Jin ◽

...

Keyword(s):

Deep Learning ◽

Large Scale ◽

Spatial Organization ◽

Domain Adaptation ◽

Single Cells ◽

Supplementary Information ◽

Target Domain ◽

Source Domain ◽

Cellular Processes ◽

Cross Domain

Abstract Motivation Cryoelectron tomography (cryo-ET) visualizes structure and spatial organization of macromolecules and their interactions with other subcellular components inside single cells in the close-to-native state at submolecular resolution. Such information is critical for the accurate understanding of cellular processes. However, subtomogram classification remains one of the major challenges for the systematic recognition and recovery of the macromolecule structures in cryo-ET because of imaging limits and data quantity. Recently, deep learning has significantly improved the throughput and accuracy of large-scale subtomogram classification. However, often it is difficult to get enough high-quality annotated subtomogram data for supervised training due to the enormous expense of labeling. To tackle this problem, it is beneficial to utilize another already annotated dataset to assist the training process. However, due to the discrepancy of image intensity distribution between source domain and target domain, the model trained on subtomograms in source domain may perform poorly in predicting subtomogram classes in the target domain. Results In this article, we adapt a few shot domain adaptation method for deep learning-based cross-domain subtomogram classification. The essential idea of our method consists of two parts: (i) take full advantage of the distribution of plentiful unlabeled target domain data, and (ii) exploit the correlation between the whole source domain dataset and few labeled target domain data. Experiments conducted on simulated and real datasets show that our method achieves significant improvement on cross domain subtomogram classification compared with baseline methods. Availability and implementation Software is available online https://github.com/xulabs/aitom. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text

Unsupervised Domain Adaptation Method Based on Discriminant Sample Selection

Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University ◽

10.1051/jnwpu/20203840828 ◽

2020 ◽

Vol 38 (4) ◽

pp. 828-837

Author(s):

Linlin Wu ◽

Guohua Peng ◽

Weidong Yan

Keyword(s):

Density Estimation ◽

Classification Accuracy ◽

Domain Adaptation ◽

Sample Selection ◽

Previous Method ◽

Probability Density Estimation ◽

Target Domain ◽

Source Domain ◽

Unsupervised Domain Adaptation ◽

Adaptation Method

In order to solve the problem that low classification accuracy caused by the different distribution of training set and test set, an unsupervised domain adaptation method based on discriminant sample selection (DSS) is proposed. DSS projects the samples of different domains onto a same subspace to reduce the distribution discrepancy between the source domain and the target domain, and weights the source domain instances to make the samples more discriminant. Different from the previous method based on the probability density estimation of samples, DSS tries to obtain the sample weights by solving a quadratic programming problem, which avoids the distribution estimation of samples and can be applied to any fields without suffering from the dimensional trouble caused by high-dimensional density estimation. Finally, DSS congregates the same classes by minimizing the intra-class distance. Experimental results show that the proposed method improves the classification accuracy and robustness.

Download Full-text

Unsupervised Domain Adaptation on Reading Comprehension

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6245 ◽

2020 ◽

Vol 34 (05) ◽

pp. 7480-7487

Author(s):

Yu Cao ◽

Meng Fang ◽

Baosheng Yu ◽

Joey Tianyi Zhou

Keyword(s):

Reading Comprehension ◽

Large Scale ◽

Domain Adaptation ◽

Generalization Capability ◽

Target Domain ◽

Adversarial Learning ◽

Unsupervised Domain Adaptation ◽

Comparable Performance ◽

Benchmark Datasets ◽

Contextual Representation

Reading comprehension (RC) has been studied in a variety of datasets with the boosted performance brought by deep neural networks. However, the generalization capability of these models across different domains remains unclear. To alleviate the problem, we investigate unsupervised domain adaptation on RC, wherein a model is trained on the labeled source domain and to be applied to the target domain with only unlabeled samples. We first show that even with the powerful BERT contextual representation, a model can not generalize well from one domain to another. To solve this, we provide a novel conditional adversarial self-training method (CASe). Specifically, our approach leverages a BERT model fine-tuned on the source dataset along with the confidence filtering to generate reliable pseudo-labeled samples in the target domain for self-training. On the other hand, it further reduces domain distribution discrepancy through conditional adversarial learning across domains. Extensive experiments show our approach achieves comparable performance to supervised models on multiple large-scale benchmark datasets.

Download Full-text

CycleEmotionGAN: Emotional Semantic Consistency Preserved CycleGAN for Adapting Image Emotions

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33012620 ◽

2019 ◽

Vol 33 ◽

pp. 2620-2627 ◽

Cited By ~ 10

Author(s):

Sicheng Zhao ◽

Chuang Lin ◽

Pengfei Xu ◽

Sendong Zhao ◽

Yuchen Guo ◽

...

Keyword(s):

Large Scale ◽

Domain Adaptation ◽

Training Data ◽

Emotion Classification ◽

Target Domain ◽

Adaptation Studies ◽

Source Domain ◽

Semantic Consistency ◽

Annotation Information ◽

Image Pairs

Deep neural networks excel at learning from large-scale labeled training data, but cannot well generalize the learned knowledge to new domains or datasets. Domain adaptation studies how to transfer models trained on one labeled source domain to another sparsely labeled or unlabeled target domain. In this paper, we investigate the unsupervised domain adaptation (UDA) problem in image emotion classification. Specifically, we develop a novel cycle-consistent adversarial model, termed CycleEmotionGAN, by enforcing emotional semantic consistency while adapting images cycleconsistently. By alternately optimizing the CycleGAN loss, the emotional semantic consistency loss, and the target classification loss, CycleEmotionGAN can adapt source domain images to have similar distributions to the target domain without using aligned image pairs. Simultaneously, the annotation information of the source images is preserved. Extensive experiments are conducted on the ArtPhoto and FI datasets, and the results demonstrate that CycleEmotionGAN significantly outperforms the state-of-the-art UDA approaches.

Download Full-text