TriGAN: image-to-image translation for multi-source domain adaptation

AbstractMost domain adaptation methods consider the problem of transferring knowledge to the target domain from a single-source dataset. However, in practical applications, we typically have access to multiple sources. In this paper we propose the first approach for multi-source domain adaptation (MSDA) based on generative adversarial networks. Our method is inspired by the observation that the appearance of a given image depends on three factors: the domain, the style (characterized in terms of low-level features variations) and the content. For this reason, we propose to project the source image features onto a space where only the dependence from the content is kept, and then re-project this invariant representation onto the pixel space using the target domain and style. In this way, new labeled images can be generated which are used to train a final target classifier. We test our approach using common MSDA benchmarks, showing that it outperforms state-of-the-art methods.

Download Full-text

Data-Efficient Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Generative Adversarial Networks

Applied Sciences ◽

10.3390/app10031092 ◽

2020 ◽

Vol 10 (3) ◽

pp. 1092 ◽

Cited By ~ 2

Author(s):

Bilel Benjdira ◽

Adel Ammar ◽

Anis Koubaa ◽

Kais Ouni

Keyword(s):

Domain Adaptation ◽

Semantic Segmentation ◽

Aerial Imagery ◽

Generative Adversarial Networks ◽

Target Domain ◽

Source Domain ◽

Adversarial Networks ◽

Semantic Label ◽

Global Accuracy ◽

The Cost

Despite the significant advances noted in semantic segmentation of aerial imagery, a considerable limitation is blocking its adoption in real cases. If we test a segmentation model on a new area that is not included in its initial training set, accuracy will decrease remarkably. This is caused by the domain shift between the new targeted domain and the source domain used to train the model. In this paper, we addressed this challenge and proposed a new algorithm that uses Generative Adversarial Networks (GAN) architecture to minimize the domain shift and increase the ability of the model to work on new targeted domains. The proposed GAN architecture contains two GAN networks. The first GAN network converts the chosen image from the target domain into a semantic label. The second GAN network converts this generated semantic label into an image that belongs to the source domain but conserves the semantic map of the target image. This resulting image will be used by the semantic segmentation model to generate a better semantic label of the first chosen image. Our algorithm is tested on the ISPRS semantic segmentation dataset and improved the global accuracy by a margin up to 24% when passing from Potsdam domain to Vaihingen domain. This margin can be increased by addition of other labeled data from the target domain. To minimize the cost of supervision in the translation process, we proposed a methodology to use these labeled data efficiently.

Download Full-text

Unsupervised Domain Adaptation Using Generative Adversarial Networks for Semantic Segmentation of Aerial Images

Remote Sensing ◽

10.3390/rs11111369 ◽

2019 ◽

Vol 11 (11) ◽

pp. 1369 ◽

Cited By ~ 21

Author(s):

Bilel Benjdira ◽

Yakoub Bazi ◽

Anis Koubaa ◽

Kais Ouni

Keyword(s):

Urban Areas ◽

Traffic Management ◽

Domain Adaptation ◽

Semantic Segmentation ◽

Aerial Images ◽

Generative Adversarial Networks ◽

Target Domain ◽

Adversarial Networks ◽

Segmentation Accuracy ◽

New City

Segmenting aerial images is of great potential in surveillance and scene understanding of urban areas. It provides a mean for automatic reporting of the different events that happen in inhabited areas. This remarkably promotes public safety and traffic management applications. After the wide adoption of convolutional neural networks methods, the accuracy of semantic segmentation algorithms could easily surpass 80% if a robust dataset is provided. Despite this success, the deployment of a pretrained segmentation model to survey a new city that is not included in the training set significantly decreases accuracy. This is due to the domain shift between the source dataset on which the model is trained and the new target domain of the new city images. In this paper, we address this issue and consider the challenge of domain adaptation in semantic segmentation of aerial images. We designed an algorithm that reduces the domain shift impact using generative adversarial networks (GANs). In the experiments, we tested the proposed methodology on the International Society for Photogrammetry and Remote Sensing (ISPRS) semantic segmentation dataset and found that our method improves overall accuracy from 35% to 52% when passing from the Potsdam domain (considered as source domain) to the Vaihingen domain (considered as target domain). In addition, the method allows efficiently recovering the inverted classes due to sensor variation. In particular, it improves the average segmentation accuracy of the inverted classes due to sensor variation from 14% to 61%.

Download Full-text

One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networks

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/344 ◽

2019 ◽

Cited By ~ 1

Author(s):

Tao He ◽

Yuan-Fang Li ◽

Lianli Gao ◽

Dongxiang Zhang ◽

Jingkuan Song

Keyword(s):

Adaptive Learning ◽

Digital Data ◽

Generative Adversarial Networks ◽

Target Domain ◽

Source Domain ◽

Learning Framework ◽

Common Space ◽

Adversarial Networks ◽

Shared Space ◽

Benchmark Datasets

With the recent explosive increase of digital data, image recognition and retrieval become a critical practical application. Hashing is an effective solution to this problem, due to its low storage requirement and high query speed. However, most of past works focus on hashing in a single (source) domain. Thus, the learned hash function may not adapt well in a new (target) domain that has a large distributional difference with the source domain. In this paper, we explore an end-to-end domain adaptive learning framework that simultaneously and precisely generates discriminative hash codes and classifies target domain images. Our method encodes two domains images into a semantic common space, followed by two independent generative adversarial networks arming at crosswise reconstructing two domains’ images, reducing domain disparity and improving alignment in the shared space. We evaluate our framework on four public benchmark datasets, all of which show that our method is superior to the other state-of-the-art methods on the tasks of object recognition and image retrieval.

Download Full-text

Unsupervised multi-source domain adaptation with no observable source data

PLoS ONE ◽

10.1371/journal.pone.0253415 ◽

2021 ◽

Vol 16 (7) ◽

pp. e0253415

Author(s):

Hyunsik Jeon ◽

Seongmin Lee ◽

U Kang

Keyword(s):

Real World ◽

Domain Adaptation ◽

State Of The Art ◽

Multiple Source ◽

Multiple Sources ◽

Target Domain ◽

Source Domain ◽

Source Data ◽

Real World Datasets ◽

Target Data

Given trained models from multiple source domains, how can we predict the labels of unlabeled data in a target domain? Unsupervised multi-source domain adaptation (UMDA) aims for predicting the labels of unlabeled target data by transferring the knowledge of multiple source domains. UMDA is a crucial problem in many real-world scenarios where no labeled target data are available. Previous approaches in UMDA assume that data are observable over all domains. However, source data are not easily accessible due to privacy or confidentiality issues in a lot of practical scenarios, although classifiers learned in source domains are readily available. In this work, we target data-free UMDA where source data are not observable at all, a novel problem that has not been studied before despite being very realistic and crucial. To solve data-free UMDA, we propose DEMS (Data-free Exploitation of Multiple Sources), a novel architecture that adapts target data to source domains without exploiting any source data, and estimates the target labels by exploiting pre-trained source classifiers. Extensive experiments for data-free UMDA on real-world datasets show that DEMS provides the state-of-the-art accuracy which is up to 27.5% point higher than that of the best baseline.

Download Full-text

Exploiting Images for Video Recognition with Hierarchical Generative Adversarial Networks

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/154 ◽

2018 ◽

Cited By ~ 1

Author(s):

Feiwu Yu ◽

Xinxiao Wu ◽

Yuchao Sun ◽

Lixin Duan

Keyword(s):

Domain Adaptation ◽

Feature Representation ◽

Generative Adversarial Networks ◽

Target Domain ◽

Feature Representations ◽

Adversarial Networks ◽

Video Recognition ◽

Novel Method ◽

High Level ◽

Heterogeneous Feature

Existing deep learning methods of video recognition usually require a large number of labeled videos for training. But for a new task, videos are often unlabeled and it is also time-consuming and labor-intensive to annotate them. Instead of human annotation, we try to make use of existing fully labeled images to help recognize those videos. However, due to the problem of domain shifts and heterogeneous feature representations, the performance of classifiers trained on images may be dramatically degraded for video recognition tasks. In this paper, we propose a novel method, called Hierarchical Generative Adversarial Networks (HiGAN), to enhance recognition in videos (i.e., target domain) by transferring knowledge from images (i.e., source domain). The HiGAN model consists of a \emph{low-level} conditional GAN and a \emph{high-level} conditional GAN. By taking advantage of these two-level adversarial learning, our method is capable of learning a domain-invariant feature representation of source images and target videos. Comprehensive experiments on two challenging video recognition datasets (i.e. UCF101 and HMDB51) demonstrate the effectiveness of the proposed method when compared with the existing state-of-the-art domain adaptation methods.

Download Full-text

Domain Adaptation-Based Automatic Modulation Recognition

Scientific Programming ◽

10.1155/2021/4277061 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Tong Li ◽

Yingzhe Xiao

Keyword(s):

Network Structure ◽

Software Defined Radio ◽

Domain Adaptation ◽

Recognition Rate ◽

Modulation Recognition ◽

Target Domain ◽

Source Domain ◽

Practical Applications ◽

Automatic Modulation Recognition ◽

Real Scenario

Deep learning-based Automatic Modulation Recognition (AMR) can improve the recognition rate compared with traditional AMR methods. However, in practical applications, as training samples and real scenario signal samples have different distributions in practical applications, the recognition rate for target domain samples can deteriorate significantly. This paper proposed an unsupervised domain adaptation based AMR method, which can enhance the recognition performance by adopting labeled samples from the source domain and unlabeled samples from the target domain. The proposed method is validated through signal samples generated from the open-sourced Software Defined Radio (SDR) GNU Radio. The training dataset is composed of labeled samples in the source domain and unlabeled samples in the target domain. In the testing dataset, the samples are from the target domain to simulate the real scenario. Through the experiment, the proposed method has a recognition rate increase of about 88% under the CNN network structure and 91% under the ResNet network structure.

Download Full-text

Cycle-consistent Generative Adversarial Networks (CycleGANs) for the Non-Parallel Creation of Fake Voice Media

10.20944/preprints201906.0104.v1 ◽

2019 ◽

Author(s):

Daniel Fleury ◽

Angelica Fleury

Keyword(s):

Domain Adaptation ◽

Discrete Source ◽

Voice Conversion ◽

Generative Adversarial Networks ◽

Male Voice ◽

Target Domain ◽

Ethical Implications ◽

Adversarial Network ◽

Adversarial Networks ◽

Object Features

The upsurge of Generative Adversarial Networks (GANs) in the previous five years has led to advancements in unsupervised data manipulation, sourced feature translation, and precise input-output synthesis through a competitive optimization of the discriminator and generator networks. More specifically, the recent rise of cycle-consistent GANs enables style transfers from a discrete source (input A) to target domain (input B) by preprocessing object features for a multi-discriminative adversarial network. Traditionally, cyclical adversarial networks have been exploited for unpaired image-to-image translation and domain adaptation by determining mapped relationships between an input A graphic and an input B graphic. However, this integral mechanism of domain adaptation can be applied to the complex acoustical features of human speech. Although well-established datasets, such as the 2018 Voice Conversion Challenge repository, paved way for female-male voice transformation, cycle-GANs have rarely been re-engineered for voices outside the datasets. More critically, cycle-GANs have massive potential to extract surface-level and hidden feature to distort an input A source into a texturally unrelated target voice. By preprocessing, compressing, and packaging unique acoustical voice properties, CycleGANs can learn to decompose speech signals and implement new translation models while preserving emotion, the intent of words, rhythm, and accents. Due to the potential of CycleGAN’s autoencoder in realistic unsupervised voice-voice conversion/feature adaptation, the researchers raise the ethical implications of controlling source input A to manipulate target voice B, particularly in cases of defamation and sabotage of target B’s words. This paper analyzes the potential of cycle-consistent GANs in deceptive voice-voice conversion by manipulating interview excerpts of political candidates.

Download Full-text

Discriminative Sparse Filtering for Multi-Source Image Classification

Sensors ◽

10.3390/s20205868 ◽

2020 ◽

Vol 20 (20) ◽

pp. 5868 ◽

Cited By ~ 1

Author(s):

Chao Han ◽

Deyun Zhou ◽

Zhen Yang ◽

Yu Xie ◽

Kai Zhang

Keyword(s):

Domain Adaptation ◽

State Of The Art ◽

Source Image ◽

Unified Framework ◽

Target Domain ◽

Maximum Mean Discrepancy ◽

Source Domain ◽

Cross Domain ◽

Sparse Filtering ◽

Novel Approach

Distribution mismatch caused by various resolutions, backgrounds, etc. can be easily found in multi-sensor systems. Domain adaptation attempts to reduce such domain discrepancy by means of different measurements, e.g., maximum mean discrepancy (MMD). Despite their success, such methods often fail to guarantee the separability of learned representation. To tackle this issue, we put forward a novel approach to jointly learn both domain-shared and discriminative representations. Specifically, we model the feature discrimination explicitly for two domains. Alternating discriminant optimization is proposed to obtain discriminative features with an l2 constraint in labeled source domain and sparse filtering is introduced to capture the intrinsic structures exists in the unlabeled target domain. Finally, they are integrated in a unified framework along with MMD to align domains. Extensive experiments compared with state-of-the-art methods verify the effectiveness of our method on cross-domain tasks.

Download Full-text

Confidence Measures for Deep Learning in Domain Adaptation

Applied Sciences ◽

10.3390/app9112192 ◽

2019 ◽

Vol 9 (11) ◽

pp. 2192 ◽

Cited By ~ 2

Author(s):

Simone Bonechi ◽

Paolo Andreini ◽

Monica Bianchini ◽

Akshay Pai ◽

Franco Scarselli

Keyword(s):

Domain Adaptation ◽

Medical Image Analysis ◽

Generative Adversarial Networks ◽

Target Domain ◽

Huge Amount ◽

Confidence Measures ◽

Adversarial Networks ◽

Large Sets ◽

Learning Tasks ◽

Network Output

In recent years, Deep Neural Networks (DNNs) have led to impressive results in a wide variety of machine learning tasks, typically relying on the existence of a huge amount of supervised data. However, in many applications (e.g., bio–medical image analysis), gathering large sets of labeled data can be very difficult and costly. Unsupervised domain adaptation exploits data from a source domain, where annotations are available, to train a model able to generalize also to a target domain, where labels are unavailable. Recent research has shown that Generative Adversarial Networks (GANs) can be successfully employed for domain adaptation, although deciding when to stop learning is a major concern for GANs. In this work, we propose some confidence measures that can be used to early stop the GAN training, also showing how such measures can be employed to predict the reliability of the network output. The effectiveness of the proposed approach has been tested in two domain adaptation tasks, with very promising results.

Download Full-text

Correlation alignment with attention mechanism for unsupervised domain adaptation

Web Intelligence ◽

10.3233/web-210447 ◽

2021 ◽

pp. 1-7

Author(s):

Rong Chen ◽

Chongguang Ren

Keyword(s):

Natural Language Processing ◽

Language Processing ◽

Transfer Process ◽

Negative Transfer ◽

Domain Adaptation ◽

Attention Mechanism ◽

Target Domain ◽

Source Domain ◽

Second Order Statistics ◽

Unsupervised Domain Adaptation

Domain adaptation aims to solve the problems of lacking labels. Most existing works of domain adaptation mainly focus on aligning the feature distributions between the source and target domain. However, in the field of Natural Language Processing, some of the words in different domains convey different sentiment. Thus not all features of the source domain should be transferred, and it would cause negative transfer when aligning the untransferable features. To address this issue, we propose a Correlation Alignment with Attention mechanism for unsupervised Domain Adaptation (CAADA) model. In the model, an attention mechanism is introduced into the transfer process for domain adaptation, which can capture the positively transferable features in source and target domain. Moreover, the CORrelation ALignment (CORAL) loss is utilized to minimize the domain discrepancy by aligning the second-order statistics of the positively transferable features extracted by the attention mechanism. Extensive experiments on the Amazon review dataset demonstrate the effectiveness of CAADA method.

Download Full-text