scholarly journals Triplet Loss Network for Unsupervised Domain Adaptation

Algorithms ◽  
2019 ◽  
Vol 12 (5) ◽  
pp. 96 ◽  
Author(s):  
Imad Eddine Ibrahim Bekkouch ◽  
Youssef Youssry ◽  
Rustam Gafarov ◽  
Adil Khan ◽  
Asad Masood Khattak
Keyword(s):  
Image Classification ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Traffic Sign ◽  
Linear Discriminant ◽  
Unsupervised Domain Adaptation ◽  
Sign Recognition ◽  
Almost All

Domain adaptation is a sub-field of transfer learning that aims at bridging the dissimilarity gap between different domains by transferring and re-using the knowledge obtained in the source domain to the target domain. Many methods have been proposed to resolve this problem, using techniques such as generative adversarial networks (GAN), but the complexity of such methods makes it hard to use them in different problems, as fine-tuning such networks is usually a time-consuming task. In this paper, we propose a method for unsupervised domain adaptation that is both simple and effective. Our model (referred to as TripNet) harnesses the idea of a discriminator and Linear Discriminant Analysis (LDA) to push the encoder to generate domain-invariant features that are category-informative. At the same time, pseudo-labelling is used for the target data to train the classifier and to bring the same classes from both domains together. We evaluate TripNet against several existing, state-of-the-art methods on three image classification tasks: Digit classification (MNIST, SVHN, and USPC datasets), object recognition (Office31 dataset), and traffic sign recognition (GTSRB and Synthetic Signs datasets). Our experimental results demonstrate that (i) TripNet beats almost all existing methods (having a similar simple model like it) on all of these tasks; and (ii) for models that are significantly more complex (or hard to train) than TripNet, it even beats their performance in some cases. Hence, the results confirm the effectiveness of using TripNet for unsupervised domain adaptation in image classification.

scholarly journals Robust Convolutional Neural Networks in SRAM-based FPGAs: a Case Study in Image Classification

2021 ◽  
Vol 16 (2) ◽  
pp. 1-12
Author(s):  
Fabio Benevenuti ◽  
Fernanda Lima Kastensmidt ◽  
Ádria Barros de Oliveira ◽  
Nemitala Added ◽  
Vitor Ângelo Paulino de Aguiar ◽  
...  
Keyword(s):  
Image Classification ◽  
Hardware Implementation ◽  
Fault Injection ◽  
Fine Tuning ◽  
Traffic Sign ◽  
Sign Recognition ◽  
User Data ◽  
Floating Point Number ◽  
Xilinx Fpga

This work discusses the main aspects of vulnerability and degradation of accuracy of an image classification engine implemented into SRAM-based FPGAs under faults. The image classification engine is an all-convolutional neural-network (CNN) trained with a dataset of traffic sign recognition benchmark. The Caffe and Ristretto frameworks were used for CNN training and fine-tuning while the ZynqNet inference engine was adopted as hardware implementation on a Xilinx 28 nm SRAM-based FPGA. The CNN under test was generated using an evolutive approach based on genetic algorithm. The methodologies for qualifying this CNN under faults is presented and both heavy-ions accelerated irradiation and emulated fault injection were performed. To cross validate results from radiation and fault injection, different implementations of the same CNN were tested using reduced arithmetic precision and protection of user data by Hamming codes, in combination with configuration memory healing by the scrubbing mechanism available in Xilinx FPGA. Some of these alternative implementations increased significantly the mission time of the CNN, when compared to the original ZynqNet operating on 32 bits floating point number, and the experiment suggests areas for further improvements on the fault injection methodology in use.

scholarly journals Unsupervised Domain Adaptation for Facial Expression Recognition Using Generative Adversarial Networks

2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Xiaoqing Wang ◽  
Xiangjun Wang ◽  
Yubo Ni
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Domain Adaptation ◽  
Recognition Task ◽  
Fine Tuning ◽  
Generative Adversarial Networks ◽  
Expression Recognition ◽  
Generative Adversarial Network ◽  
Unsupervised Domain Adaptation ◽  
Adversarial Network

In the facial expression recognition task, a good-performing convolutional neural network (CNN) model trained on one dataset (source dataset) usually performs poorly on another dataset (target dataset). This is because the feature distribution of the same emotion varies in different datasets. To improve the cross-dataset accuracy of the CNN model, we introduce an unsupervised domain adaptation method, which is especially suitable for unlabelled small target dataset. In order to solve the problem of lack of samples from the target dataset, we train a generative adversarial network (GAN) on the target dataset and use the GAN generated samples to fine-tune the model pretrained on the source dataset. In the process of fine-tuning, we give the unlabelled GAN generated samples distributed pseudolabels dynamically according to the current prediction probabilities. Our method can be easily applied to any existing convolutional neural networks (CNN). We demonstrate the effectiveness of our method on four facial expression recognition datasets with two CNN structures and obtain inspiring results.

Unsupervised domain adaptation based on cluster matching and Fisher criterion for image classification

2021 ◽  
Vol 91 ◽  
pp. 107041
Author(s):  
Heyou Chang ◽  
Fanlong Zhang ◽  
Shuai Ma ◽  
Guangwei Gao ◽  
Hao Zheng ◽  
...  
Keyword(s):  
Image Classification ◽  
Domain Adaptation ◽  
Fisher Criterion ◽  
Unsupervised Domain Adaptation ◽  
Cluster Matching

Correlation alignment with attention mechanism for unsupervised domain adaptation

2021 ◽  
pp. 1-7
Author(s):  
Rong Chen ◽  
Chongguang Ren
Keyword(s):  
Natural Language Processing ◽  
Language Processing ◽  
Transfer Process ◽  
Negative Transfer ◽  
Domain Adaptation ◽  
Attention Mechanism ◽  
Target Domain ◽  
Source Domain ◽  
Second Order Statistics ◽  
Unsupervised Domain Adaptation

Domain adaptation aims to solve the problems of lacking labels. Most existing works of domain adaptation mainly focus on aligning the feature distributions between the source and target domain. However, in the field of Natural Language Processing, some of the words in different domains convey different sentiment. Thus not all features of the source domain should be transferred, and it would cause negative transfer when aligning the untransferable features. To address this issue, we propose a Correlation Alignment with Attention mechanism for unsupervised Domain Adaptation (CAADA) model. In the model, an attention mechanism is introduced into the transfer process for domain adaptation, which can capture the positively transferable features in source and target domain. Moreover, the CORrelation ALignment (CORAL) loss is utilized to minimize the domain discrepancy by aligning the second-order statistics of the positively transferable features extracted by the attention mechanism. Extensive experiments on the Amazon review dataset demonstrate the effectiveness of CAADA method.

scholarly journals TSAL: Two steps Adversarial learning based domain adaptation

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Haidi Hasan Badr ◽  
Nayer Mahmoud Wanas ◽  
Magda Fayek
Keyword(s):  
Network Architecture ◽  
Domain Adaptation ◽  
Data Availability ◽  
Fine Tuning ◽  
Target Domain ◽  
Adversarial Learning ◽  
Prior Art ◽  
Real World Problem ◽  
Wide Range ◽  
Benchmark Datasets

Since labeled data availability differs greatly across domains, Domain Adaptation focuses on learning in new and unfamiliar domains by reducing distribution divergence. Recent research suggests that the adversarial learning approach could be a promising way to achieve the domain adaptation objective. Adversarial learning is a strategy for learning domain-transferable features in robust deep networks. This paper introduces the TSAL paradigm, a two-step adversarial learning framework. It addresses the real-world problem of text classification, where source domain(s) has labeled data but target domain (s) has only unlabeled data. TSAL utilizes joint adversarial learning with class information and domain alignment deep network architecture to learn both domain-invariant and domain-specific features extractors. It consists of two training steps that are similar to the paradigm, in which pre-trained model weights are used as initialization for training with new data. TSAL’s two training phases, however, are based on the same data, not different data, as is the case with fine-tuning. Furthermore, TSAL only uses the learned domain-invariant feature extractor from the first training as an initialization for its peer in subsequent training. By doubling the training, TSAL can emphasize the leverage of the small unlabeled target domain and learn effectively what to share between various domains. A detailed analysis of many benchmark datasets reveals that our model consistently outperforms the prior art across a wide range of dataset distributions.

scholarly journals Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

2021 ◽  
Author(s):  
Jiahao Fan ◽  
Hangyu Zhu ◽  
Xinyu Jiang ◽  
Long Meng ◽  
Cong Fu ◽  
...  
Keyword(s):  
Large Scale ◽  
Domain Adaptation ◽  
Source Model ◽  
Deep Sleep ◽  
Sleep Staging ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Generalization Problem ◽  
Source Models

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.<br>

scholarly journals Does BERT need domain adaptation for clinical negation detection?

2020 ◽  
Vol 27 (4) ◽  
pp. 584-591 ◽  
Author(s):  
Chen Lin ◽  
Steven Bethard ◽  
Dmitriy Dligach ◽  
Farig Sadeque ◽  
Guergana Savova ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Adaptation Algorithm ◽  
Learning Methods ◽  
Clinical Text ◽  
Unsupervised Domain Adaptation ◽  
Adversarial Training ◽  
Negation Detection ◽  
Adaptation Method

Abstract Introduction Classifying whether concepts in an unstructured clinical text are negated is an important unsolved task. New domain adaptation and transfer learning methods can potentially address this issue. Objective We examine neural unsupervised domain adaptation methods, introducing a novel combination of domain adaptation with transformer-based transfer learning methods to improve negation detection. We also want to better understand the interaction between the widely used bidirectional encoder representations from transformers (BERT) system and domain adaptation methods. Materials and Methods We use 4 clinical text datasets that are annotated with negation status. We evaluate a neural unsupervised domain adaptation algorithm and BERT, a transformer-based model that is pretrained on massive general text datasets. We develop an extension to BERT that uses domain adversarial training, a neural domain adaptation method that adds an objective to the negation task, that the classifier should not be able to distinguish between instances from 2 different domains. Results The domain adaptation methods we describe show positive results, but, on average, the best performance is obtained by plain BERT (without the extension). We provide evidence that the gains from BERT are likely not additive with the gains from domain adaptation. Discussion Our results suggest that, at least for the task of clinical negation detection, BERT subsumes domain adaptation, implying that BERT is already learning very general representations of negation phenomena such that fine-tuning even on a specific corpus does not lead to much overfitting. Conclusion Despite being trained on nonclinical text, the large training sets of models like BERT lead to large gains in performance for the clinical negation detection task.

ITERATIVE SELF-LABELING DOMAIN ADAPTATION FOR LINEAR STRUCTURED IMAGE CLASSIFICATION

2013 ◽  
Vol 22 (05) ◽  
pp. 1360005 ◽  
Author(s):  
AMAURY HABRARD ◽  
JEAN-PHILIPPE PEYRACHE ◽  
MARC SEBBAN
Keyword(s):  
Image Classification ◽  
Domain Adaptation ◽  
Research Area ◽  
Target Domain ◽  
Sparse Models ◽  
Similarity Functions ◽  
Generalization Bounds ◽  
Source Data ◽  
New Research ◽  
Target Data

A strong assumption to derive generalization guarantees in the standard PAC framework is that training (or source) data and test (or target) data are drawn according to the same distribution. Because of the presence of possibly outdated data in the training set, or the use of biased collections, this assumption is often violated in real-world applications leading to different source and target distributions. To go around this problem, a new research area known as Domain Adaptation (DA) has recently been introduced giving rise to many adaptation algorithms and theoretical results in the form of generalization bounds. This paper deals with self-labeling DA whose goal is to iteratively incorporate semi-labeled target data in the learning set to progressively adapt the classifier from the source to the target domain. The contribution of this work is three-fold: First, we provide the minimum and necessary theoretical conditions for a self-labeling DA algorithm to perform an actual domain adaptation. Second, following these theoretical recommendations, we design a new iterative DA algorithm, called GESIDA, able to deal with structured data. This algorithm makes use of the new theory of learning with (ε,γ,τ)-good similarity functions introduced by Balcan et al., which does not require the use of a valid kernel to learn well and allows us to induce sparse models. Finally, we apply our algorithm on a structured image classification task and show that self-labeling domain adaptation is a new original way to deal with scaling and rotation problems.

Sign in / Sign up

Export Citation Format

Share Document