Intramodality Domain Adaptation Using Self Ensembling and Adversarial Training

2019 ◽  
pp. 28-36
Author(s):  
Zahil Shanis ◽  
Samuel Gerber ◽  
Mingchen Gao ◽  
Andinet Enquobahrie
Keyword(s):  
Domain Adaptation ◽  
Adversarial Training

A new deep domain adaptation method with joint adversarial training for online detection of bearing early fault

2021 ◽  
Author(s):  
Wentao Mao ◽  
Ling Ding ◽  
Yamin Liu ◽  
Sajad Saraygord Afshari ◽  
Xihui Liang
Keyword(s):  
Domain Adaptation ◽  
Online Detection ◽  
Adversarial Training ◽  
Adaptation Method

scholarly journals Domain-Adversarial Training of Self-Attention-Based Networks for Land Cover Classification Using Multi-Temporal Sentinel-2 Satellite Imagery

2021 ◽  
Vol 13 (13) ◽  
pp. 2564
Author(s):  
Mauro Martini ◽  
Vittorio Mazzia ◽  
Aleem Khaliq ◽  
Marcello Chiaberge
Keyword(s):  
Land Cover ◽  
Large Scale ◽  
Domain Adaptation ◽  
Learning Approaches ◽  
Atmospheric Conditions ◽  
Practical Applications ◽  
Area Of Interest ◽  
Multi Temporal ◽  
Significant Performance ◽  
Adversarial Training

The increasing availability of large-scale remote sensing labeled data has prompted researchers to develop increasingly precise and accurate data-driven models for land cover and crop classification (LC&CC). Moreover, with the introduction of self-attention and introspection mechanisms, deep learning approaches have shown promising results in processing long temporal sequences in the multi-spectral domain with a contained computational request. Nevertheless, most practical applications cannot rely on labeled data, and in the field, surveys are a time-consuming solution that pose strict limitations to the number of collected samples. Moreover, atmospheric conditions and specific geographical region characteristics constitute a relevant domain gap that does not allow direct applicability of a trained model on the available dataset to the area of interest. In this paper, we investigate adversarial training of deep neural networks to bridge the domain discrepancy between distinct geographical zones. In particular, we perform a thorough analysis of domain adaptation applied to challenging multi-spectral, multi-temporal data, accurately highlighting the advantages of adapting state-of-the-art self-attention-based models for LC&CC to different target zones where labeled data are not available. Extensive experimentation demonstrated significant performance and generalization gain in applying domain-adversarial training to source and target regions with marked dissimilarities between the distribution of extracted features.

scholarly journals Does BERT need domain adaptation for clinical negation detection?

2020 ◽  
Vol 27 (4) ◽  
pp. 584-591 ◽  
Author(s):  
Chen Lin ◽  
Steven Bethard ◽  
Dmitriy Dligach ◽  
Farig Sadeque ◽  
Guergana Savova ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Adaptation Algorithm ◽  
Learning Methods ◽  
Clinical Text ◽  
Unsupervised Domain Adaptation ◽  
Adversarial Training ◽  
Negation Detection ◽  
Adaptation Method

Abstract Introduction Classifying whether concepts in an unstructured clinical text are negated is an important unsolved task. New domain adaptation and transfer learning methods can potentially address this issue. Objective We examine neural unsupervised domain adaptation methods, introducing a novel combination of domain adaptation with transformer-based transfer learning methods to improve negation detection. We also want to better understand the interaction between the widely used bidirectional encoder representations from transformers (BERT) system and domain adaptation methods. Materials and Methods We use 4 clinical text datasets that are annotated with negation status. We evaluate a neural unsupervised domain adaptation algorithm and BERT, a transformer-based model that is pretrained on massive general text datasets. We develop an extension to BERT that uses domain adversarial training, a neural domain adaptation method that adds an objective to the negation task, that the classifier should not be able to distinguish between instances from 2 different domains. Results The domain adaptation methods we describe show positive results, but, on average, the best performance is obtained by plain BERT (without the extension). We provide evidence that the gains from BERT are likely not additive with the gains from domain adaptation. Discussion Our results suggest that, at least for the task of clinical negation detection, BERT subsumes domain adaptation, implying that BERT is already learning very general representations of negation phenomena such that fine-tuning even on a specific corpus does not lead to much overfitting. Conclusion Despite being trained on nonclinical text, the large training sets of models like BERT lead to large gains in performance for the clinical negation detection task.

scholarly journals Aspect-augmented Adversarial Networks for Domain Adaptation

2017 ◽  
Vol 5 ◽  
pp. 515-528 ◽  
Author(s):  
Yuan Zhang ◽  
Regina Barzilay ◽  
Tommi Jaakkola
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Experimental Results ◽  
Dependent Manner ◽  
Target Classification ◽  
Adversarial Networks ◽  
Adversarial Training ◽  
Classification Tasks ◽  
Class Labels

We introduce a neural method for transfer learning between two (source and target) classification tasks or aspects over the same domain. Rather than training on target labels, we use a few keywords pertaining to source and target aspects indicating sentence relevance instead of document class labels. Documents are encoded by learning to embed and softly select relevant sentences in an aspect-dependent manner. A shared classifier is trained on the source encoded documents and labels, and applied to target encoded documents. We ensure transfer through aspect-adversarial training so that encoded documents are, as sets, aspect-invariant. Experimental results demonstrate that our approach outperforms different baselines and model variants on two datasets, yielding an improvement of 27% on a pathology dataset and 5% on a review dataset.

scholarly journals Domain Adaptation with Adversarial Training and Graph Embeddings

2018 ◽  
Author(s):  
Firoj Alam ◽  
Shafiq Joty ◽  
Muhammad Imran
Keyword(s):  
Domain Adaptation ◽  
Graph Embeddings ◽  
Adversarial Training

scholarly journals Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks

2021 ◽  
Vol 11 (18) ◽  
pp. 8412
Author(s):  
Hyeong-Ju Na ◽  
Jeong-Sik Park
Keyword(s):  
Speech Recognition ◽  
Domain Adaptation ◽  
Training Data ◽  
Baseline Model ◽  
Linguistic Differences ◽  
Computational Costs ◽  
Accented Speech ◽  
Feature Extractor ◽  
Adversarial Training ◽  
End To End

The performance of automatic speech recognition (ASR) may be degraded when accented speech is recognized because the speech has some linguistic differences from standard speech. Conventional accented speech recognition studies have utilized the accent embedding method, in which the accent embedding features are directly fed into the ASR network. Although the method improves the performance of accented speech recognition, it has some restrictions, such as increasing the computational costs. This study proposes an efficient method of training the ASR model for accented speech in a domain adversarial way based on the Domain Adversarial Neural Network (DANN). The DANN plays a role as a domain adaptation in which the training data and test data have different distributions. Thus, our approach is expected to construct a reliable ASR model for accented speech by reducing the distribution differences between accented speech and standard speech. DANN has three sub-networks: the feature extractor, the domain classifier, and the label predictor. To adjust the DANN for accented speech recognition, we constructed these three sub-networks independently, considering the characteristics of accented speech. In particular, we used an end-to-end framework based on Connectionist Temporal Classification (CTC) to develop the label predictor, a very important module that directly affects ASR results. To verify the efficiency of the proposed approach, we conducted several experiments of accented speech recognition for four English accents including Australian, Canadian, British (England), and Indian accents. The experimental results showed that the proposed DANN-based model outperformed the baseline model for all accents, indicating that the end-to-end domain adversarial training effectively reduced the distribution differences between accented speech and standard speech.

scholarly journals Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis

2020 ◽  
Vol 34 (05) ◽  
pp. 7618-7625
Author(s):  
Yong Dai ◽  
Jian Liu ◽  
Xiancong Ren ◽  
Zenglin Xu
Keyword(s):  
Sentiment Analysis ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Weak Assumption ◽  
Target Domain ◽  
Smoothness Assumption ◽  
Unsupervised Domain Adaptation ◽  
Good Target ◽  
Adversarial Training ◽  
Learning Frameworks

Multi-source unsupervised domain adaptation (MS-UDA) for sentiment analysis (SA) aims to leverage useful information in multiple source domains to help do SA in an unlabeled target domain that has no supervised information. Existing algorithms of MS-UDA either only exploit the shared features, i.e., the domain-invariant information, or based on some weak assumption in NLP, e.g., smoothness assumption. To avoid these problems, we propose two transfer learning frameworks based on the multi-source domain adaptation methodology for SA by combining the source hypotheses to derive a good target hypothesis. The key feature of the first framework is a novel Weighting Scheme based Unsupervised Domain Adaptation framework ((WS-UDA), which combine the source classifiers to acquire pseudo labels for target instances directly. While the second framework is a Two-Stage Training based Unsupervised Domain Adaptation framework (2ST-UDA), which further exploits these pseudo labels to train a target private extractor. Importantly, the weights assigned to each source classifier are based on the relations between target instances and source domains, which measured by a discriminator through the adversarial training. Furthermore, through the same discriminator, we also fulfill the separation of shared features and private features.Experimental results on two SA datasets demonstrate the promising performance of our frameworks, which outperforms unsupervised state-of-the-art competitors.

Sign in / Sign up

Export Citation Format

Share Document