scholarly journals An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation

2020 ◽  
Vol 34 (07) ◽  
pp. 12613-12620 ◽  
Author(s):  
Jihan Yang ◽  
Ruijia Xu ◽  
Ruiyu Li ◽  
Xiaojuan Qi ◽  
Xiaoyong Shen ◽  
...  
Keyword(s):  
Marginal Distribution ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Feature Space ◽  
Semantic Segmentation ◽  
The State ◽  
Object Size ◽  
Feature Maps ◽  
Feature Representations ◽  
Unsupervised Domain Adaptation

We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 → Cityscapes and SYNTHIA → Cityscapes.

scholarly journals Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation

2019 ◽  
Vol 33 ◽  
pp. 5581-5588 ◽  
Author(s):  
Yonghao Xu ◽  
Bo Du ◽  
Lefei Zhang ◽  
Qian Zhang ◽  
Guoli Wang ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Great Success ◽  
Learning Models ◽  
Target Domain ◽  
Attention Networks ◽  
Source Domain ◽  
Benchmark Datasets ◽  
Different Levels

Recent years have witnessed the great success of deep learning models in semantic segmentation. Nevertheless, these models may not generalize well to unseen image domains due to the phenomenon of domain shift. Since pixel-level annotations are laborious to collect, developing algorithms which can adapt labeled data from source domain to target domain is of great significance. To this end, we propose self-ensembling attention networks to reduce the domain gap between different datasets. To the best of our knowledge, the proposed method is the first attempt to introduce selfensembling model to domain adaptation for semantic segmentation, which provides a different view on how to learn domain-invariant features. Besides, since different regions in the image usually correspond to different levels of domain gap, we introduce the attention mechanism into the proposed framework to generate attention-aware features, which are further utilized to guide the calculation of consistency loss in the target domain. Experiments on two benchmark datasets demonstrate that the proposed framework can yield competitive performance compared with the state of the art methods.

scholarly journals Simultaneously Improve Transferability and Discriminability for Adversarial Domain Adaptation

Entropy ◽  
2021 ◽  
Vol 24 (1) ◽  
pp. 44
Author(s):  
Ting Xiao ◽  
Cangning Fan ◽  
Peng Liu ◽  
Hongwei Liu
Keyword(s):  
Conditional Distribution ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Feature Space ◽  
Low Rank ◽  
Computationally Efficient ◽  
Adversarial Learning ◽  
Joint Distributions ◽  
Distribution Matching ◽  
Maximum Separation

Although adversarial domain adaptation enhances feature transferability, the feature discriminability will be degraded in the process of adversarial learning. Moreover, most domain adaptation methods only focus on distribution matching in the feature space; however, shifts in the joint distributions of input features and output labels linger in the network, and thus, the transferability is not fully exploited. In this paper, we propose a matrix rank embedding (MRE) method to enhance feature discriminability and transferability simultaneously. MRE restores a low-rank structure for data in the same class and enforces a maximum separation structure for data in different classes. In this manner, the variations within the subspace are reduced, and the separation between the subspaces is increased, resulting in improved discriminability. In addition to statistically aligning the class-conditional distribution in the feature space, MRE forces the data of the same class in different domains to exhibit an approximate low-rank structure, thereby aligning the class-conditional distribution in the label space, resulting in improved transferability. MRE is computationally efficient and can be used as a plug-and-play term for other adversarial domain adaptation networks. Comprehensive experiments demonstrate that MRE can advance state-of-the-art domain adaptation methods.

scholarly journals Unsupervised Domain Adaptation in Semantic Segmentation: A Review

Technologies ◽  
2020 ◽  
Vol 8 (2) ◽  
pp. 35
Author(s):  
Marco Toldo ◽  
Andrea Maracani ◽  
Umberto Michieli ◽  
Pietro Zanuttigh
Keyword(s):  
Ad Hoc ◽  
Domain Adaptation ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Input Image ◽  
Comprehensive Overview ◽  
Open Problems ◽  
Unsupervised Domain Adaptation ◽  
Driving Scenario ◽  
Segmentation Models

The aim of this paper is to give an overview of the recent advancements in the Unsupervised Domain Adaptation (UDA) of deep networks for semantic segmentation. This task is attracting a wide interest since semantic segmentation models require a huge amount of labeled data and the lack of data fitting specific requirements is the main limitation in the deployment of these techniques. This field has been recently explored and has rapidly grown with a large number of ad-hoc approaches. This motivates us to build a comprehensive overview of the proposed methodologies and to provide a clear categorization. In this paper, we start by introducing the problem, its formulation and the various scenarios that can be considered. Then, we introduce the different levels at which adaptation strategies may be applied: namely, at the input (image) level, at the internal features representation and at the output level. Furthermore, we present a detailed overview of the literature in the field, dividing previous methods based on the following (non mutually exclusive) categories: adversarial learning, generative-based, analysis of the classifier discrepancies, self-teaching, entropy minimization, curriculum learning and multi-task learning. Novel research directions are also briefly introduced to give a hint of interesting open problems in the field. Finally, a comparison of the performance of the various methods in the widely used autonomous driving scenario is presented.

scholarly journals Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis

2020 ◽  
Vol 34 (05) ◽  
pp. 7618-7625
Author(s):  
Yong Dai ◽  
Jian Liu ◽  
Xiancong Ren ◽  
Zenglin Xu
Keyword(s):  
Sentiment Analysis ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Weak Assumption ◽  
Target Domain ◽  
Smoothness Assumption ◽  
Unsupervised Domain Adaptation ◽  
Good Target ◽  
Adversarial Training ◽  
Learning Frameworks

Multi-source unsupervised domain adaptation (MS-UDA) for sentiment analysis (SA) aims to leverage useful information in multiple source domains to help do SA in an unlabeled target domain that has no supervised information. Existing algorithms of MS-UDA either only exploit the shared features, i.e., the domain-invariant information, or based on some weak assumption in NLP, e.g., smoothness assumption. To avoid these problems, we propose two transfer learning frameworks based on the multi-source domain adaptation methodology for SA by combining the source hypotheses to derive a good target hypothesis. The key feature of the first framework is a novel Weighting Scheme based Unsupervised Domain Adaptation framework ((WS-UDA), which combine the source classifiers to acquire pseudo labels for target instances directly. While the second framework is a Two-Stage Training based Unsupervised Domain Adaptation framework (2ST-UDA), which further exploits these pseudo labels to train a target private extractor. Importantly, the weights assigned to each source classifier are based on the relations between target instances and source domains, which measured by a discriminator through the adversarial training. Furthermore, through the same discriminator, we also fulfill the separation of shared features and private features.Experimental results on two SA datasets demonstrate the promising performance of our frameworks, which outperforms unsupervised state-of-the-art competitors.

scholarly journals Semantics-Aligned Representation Learning for Person Re-Identification

2020 ◽  
Vol 34 (07) ◽  
pp. 11173-11180 ◽  
Author(s):  
Xin Jin ◽  
Cuiling Lan ◽  
Wenjun Zeng ◽  
Guoqiang Wei ◽  
Zhibo Chen
Keyword(s):  
State Of The Art ◽  
Representation Learning ◽  
The State ◽  
Feature Representation ◽  
Texture Image ◽  
Computationally Efficient ◽  
Feature Maps ◽  
Benchmark Datasets ◽  
Texture Generation ◽  
Base Network

Person re-identification (reID) aims to match person images to retrieve the ones with the same identity. This is a challenging task, as the images to be matched are generally semantically misaligned due to the diversity of human poses and capture viewpoints, incompleteness of the visible bodies (due to occlusion), etc. In this paper, we propose a framework that drives the reID network to learn semantics-aligned feature representation through delicate supervision designs. Specifically, we build a Semantics Aligning Network (SAN) which consists of a base network as encoder (SA-Enc) for re-ID, and a decoder (SA-Dec) for reconstructing/regressing the densely semantics aligned full texture image. We jointly train the SAN under the supervisions of person re-identification and aligned texture generation. Moreover, at the decoder, besides the reconstruction loss, we add Triplet ReID constraints over the feature maps as the perceptual losses. The decoder is discarded in the inference and thus our scheme is computationally efficient. Ablation studies demonstrate the effectiveness of our design. We achieve the state-of-the-art performances on the benchmark datasets CUHK03, Market1501, MSMT17, and the partial person reID dataset Partial REID.

scholarly journals TET-GAN: Text Effects Transfer via Stylization and Destylization

2019 ◽  
Vol 33 ◽  
pp. 1238-1245 ◽  
Author(s):  
Shuai Yang ◽  
Jiaying Liu ◽  
Wenjing Wang ◽  
Zongming Guo
Keyword(s):  
Network Design ◽  
State Of The Art ◽  
The State ◽  
High Quality ◽  
Style Transfer ◽  
Feature Representations ◽  
Art Methods ◽  
Matching Criteria

Text effects transfer technology automatically makes the text dramatically more impressive. However, previous style transfer methods either study the model for general style, which cannot handle the highly-structured text effects along the glyph, or require manual design of subtle matching criteria for text effects. In this paper, we focus on the use of the powerful representation abilities of deep neural features for text effects transfer. For this purpose, we propose a novel Texture Effects Transfer GAN (TET-GAN), which consists of a stylization subnetwork and a destylization subnetwork. The key idea is to train our network to accomplish both the objective of style transfer and style removal, so that it can learn to disentangle and recombine the content and style features of text effects images. To support the training of our network, we propose a new text effects dataset with as much as 64 professionally designed styles on 837 characters. We show that the disentangled feature representations enable us to transfer or remove all these styles on arbitrary glyphs using one network. Furthermore, the flexible network design empowers TET-GAN to efficiently extend to a new text style via oneshot learning where only one example is required. We demonstrate the superiority of the proposed method in generating high-quality stylized text over the state-of-the-art methods.

scholarly journals Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling

2020 ◽  
Vol 34 (04) ◽  
pp. 6243-6250 ◽  
Author(s):  
Qian Wang ◽  
Toby Breckon
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Structured Prediction ◽  
Target Domain ◽  
Source Domain ◽  
Domain Specific ◽  
Unsupervised Domain Adaptation ◽  
Deep Feature ◽  
Significant Performance ◽  
Error Accumulation

Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain suffer from significant performance drop when directly applied to the samples from the target domain. To address this issue, different approaches have been proposed to learn domain-invariant features or domain-specific classifiers. In either case, the lack of labeled samples in the target domain can be an issue which is usually overcome by pseudo-labeling. Inaccurate pseudo-labeling, however, could result in catastrophic error accumulation during learning. In this paper, we propose a novel selective pseudo-labeling strategy based on structured prediction. The idea of structured prediction is inspired by the fact that samples in the target domain are well clustered within the deep feature space so that unsupervised clustering analysis can be used to facilitate accurate pseudo-labeling. Experimental results on four datasets (i.e. Office-Caltech, Office31, ImageCLEF-DA and Office-Home) validate our approach outperforms contemporary state-of-the-art methods.

scholarly journals Domain Agnostic Real-Valued Specificity Prediction

2019 ◽  
Vol 33 ◽  
pp. 6610-6617 ◽  
Author(s):  
Wei-Jen Ko ◽  
Greg Durrett ◽  
Junyi Jessy Li
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Mean Absolute Error ◽  
Absolute Error ◽  
Error Reduction ◽  
Level Of Detail ◽  
Reference Distribution ◽  
Unsupervised Domain Adaptation ◽  
Current State ◽  
Prediction Systems

Sentence specificity quantifies the level of detail in a sentence, characterizing the organization of information in discourse. While this information is useful for many downstream applications, specificity prediction systems predict very coarse labels (binary or ternary) and are trained on and tailored toward specific domains (e.g., news). The goal of this work is to generalize specificity prediction to domains where no labeled data is available and output more nuanced realvalued specificity ratings.We present an unsupervised domain adaptation system for sentence specificity prediction, specifically designed to output real-valued estimates from binary training labels. To calibrate the values of these predictions appropriately, we regularize the posterior distribution of the labels towards a reference distribution. We show that our framework generalizes well to three different domains with 50%-68% mean absolute error reduction than the current state-of-the-art system trained for news sentence specificity. We also demonstrate the potential of our work in improving the quality and informativeness of dialogue generation systems.

Review the state-of-the-art technologies of semantic segmentation based on deep learning

2022 ◽  
Author(s):  
Yujian Mo ◽  
Yan Wu ◽  
Xinneng Yang ◽  
Feilin Liu ◽  
Yujun Liao
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
The State
Sign in / Sign up

Export Citation Format

Share Document