Unsupervised Domain Adaptation for Dysarthric Speech Detection via Domain Adversarial Training and Mutual Information Minimization

Abstract Introduction Classifying whether concepts in an unstructured clinical text are negated is an important unsolved task. New domain adaptation and transfer learning methods can potentially address this issue. Objective We examine neural unsupervised domain adaptation methods, introducing a novel combination of domain adaptation with transformer-based transfer learning methods to improve negation detection. We also want to better understand the interaction between the widely used bidirectional encoder representations from transformers (BERT) system and domain adaptation methods. Materials and Methods We use 4 clinical text datasets that are annotated with negation status. We evaluate a neural unsupervised domain adaptation algorithm and BERT, a transformer-based model that is pretrained on massive general text datasets. We develop an extension to BERT that uses domain adversarial training, a neural domain adaptation method that adds an objective to the negation task, that the classifier should not be able to distinguish between instances from 2 different domains. Results The domain adaptation methods we describe show positive results, but, on average, the best performance is obtained by plain BERT (without the extension). We provide evidence that the gains from BERT are likely not additive with the gains from domain adaptation. Discussion Our results suggest that, at least for the task of clinical negation detection, BERT subsumes domain adaptation, implying that BERT is already learning very general representations of negation phenomena such that fine-tuning even on a specific corpus does not lead to much overfitting. Conclusion Despite being trained on nonclinical text, the large training sets of models like BERT lead to large gains in performance for the clinical negation detection task.

Download Full-text

Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6262 ◽

2020 ◽

Vol 34 (05) ◽

pp. 7618-7625

Author(s):

Yong Dai ◽

Jian Liu ◽

Xiancong Ren ◽

Zenglin Xu

Keyword(s):

Sentiment Analysis ◽

Domain Adaptation ◽

State Of The Art ◽

Weak Assumption ◽

Target Domain ◽

Smoothness Assumption ◽

Unsupervised Domain Adaptation ◽

Good Target ◽

Adversarial Training ◽

Learning Frameworks

Multi-source unsupervised domain adaptation (MS-UDA) for sentiment analysis (SA) aims to leverage useful information in multiple source domains to help do SA in an unlabeled target domain that has no supervised information. Existing algorithms of MS-UDA either only exploit the shared features, i.e., the domain-invariant information, or based on some weak assumption in NLP, e.g., smoothness assumption. To avoid these problems, we propose two transfer learning frameworks based on the multi-source domain adaptation methodology for SA by combining the source hypotheses to derive a good target hypothesis. The key feature of the first framework is a novel Weighting Scheme based Unsupervised Domain Adaptation framework ((WS-UDA), which combine the source classifiers to acquire pseudo labels for target instances directly. While the second framework is a Two-Stage Training based Unsupervised Domain Adaptation framework (2ST-UDA), which further exploits these pseudo labels to train a target private extractor. Importantly, the weights assigned to each source classifier are based on the relations between target instances and source domains, which measured by a discriminator through the adversarial training. Furthermore, through the same discriminator, we also fulfill the separation of shared features and private features.Experimental results on two SA datasets demonstrate the promising performance of our frameworks, which outperforms unsupervised state-of-the-art competitors.

Download Full-text

Unsupervised Domain Adaptation via Domain Adversarial Training for Speaker Recognition

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ◽

10.1109/icassp.2018.8461423 ◽

2018 ◽

Cited By ~ 23

Author(s):

Qing Wang ◽

Wei Rao ◽

Sining Sun ◽

Leib Xie ◽

Eng Siong Chng ◽

...

Keyword(s):

Speaker Recognition ◽

Domain Adaptation ◽

Unsupervised Domain Adaptation ◽

Adversarial Training

Download Full-text

Adapting instance weights for unsupervised domain adaptation using quadratic mutual information and subspace learning

2016 23rd International Conference on Pattern Recognition (ICPR) ◽

10.1109/icpr.2016.7899859 ◽

2016 ◽

Cited By ~ 1

Author(s):

M.N.A. Khan ◽

Douglas R. Heisterkamp

Keyword(s):

Mutual Information ◽

Domain Adaptation ◽

Subspace Learning ◽

Unsupervised Domain Adaptation

Download Full-text

Bidirectional Adversarial Training for Semi-Supervised Domain Adaptation

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/130 ◽

2020 ◽

Author(s):

Pin Jiang ◽

Aming Wu ◽

Yahong Han ◽

Yunfeng Shao ◽

Meiyu Qi ◽

...

Keyword(s):

Additional Data ◽

Domain Adaptation ◽

State Of The Art ◽

Powerful Method ◽

Target Domain ◽

Unsupervised Domain Adaptation ◽

Benchmark Datasets ◽

Adversarial Examples ◽

Adversarial Training ◽

Effective Use

Semi-supervised domain adaptation (SSDA) is a novel branch of machine learning that scarce labeled target examples are available, compared with unsupervised domain adaptation. To make effective use of these additional data so as to bridge the domain gap, one possible way is to generate adversarial examples, which are images with additional perturbations, between the two domains and fill the domain gap. Adversarial training has been proven to be a powerful method for this purpose. However, the traditional adversarial training adds noises in arbitrary directions, which is inefficient to migrate between domains, or generate directional noises from the source to target domain and reverse. In this work, we devise a general bidirectional adversarial training method and employ gradient to guide adversarial examples across the domain gap, i.e., the Adaptive Adversarial Training (AAT) for source to target domain and Entropy-penalized Virtual Adversarial Training (E-VAT) for target to source domain. Particularly, we devise a Bidirectional Adversarial Training (BiAT) network to perform diverse adversarial trainings jointly. We evaluate the effectiveness of BiAT on three benchmark datasets and experimental results demonstrate the proposed method achieves the state-of-the-art.

Download Full-text