scholarly journals Data-Efficient Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Generative Adversarial Networks

2020 ◽  
Vol 10 (3) ◽  
pp. 1092 ◽  
Author(s):  
Bilel Benjdira ◽  
Adel Ammar ◽  
Anis Koubaa ◽  
Kais Ouni
Keyword(s):  
Domain Adaptation ◽  
Semantic Segmentation ◽  
Aerial Imagery ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Source Domain ◽  
Adversarial Networks ◽  
Semantic Label ◽  
Global Accuracy ◽  
The Cost

Despite the significant advances noted in semantic segmentation of aerial imagery, a considerable limitation is blocking its adoption in real cases. If we test a segmentation model on a new area that is not included in its initial training set, accuracy will decrease remarkably. This is caused by the domain shift between the new targeted domain and the source domain used to train the model. In this paper, we addressed this challenge and proposed a new algorithm that uses Generative Adversarial Networks (GAN) architecture to minimize the domain shift and increase the ability of the model to work on new targeted domains. The proposed GAN architecture contains two GAN networks. The first GAN network converts the chosen image from the target domain into a semantic label. The second GAN network converts this generated semantic label into an image that belongs to the source domain but conserves the semantic map of the target image. This resulting image will be used by the semantic segmentation model to generate a better semantic label of the first chosen image. Our algorithm is tested on the ISPRS semantic segmentation dataset and improved the global accuracy by a margin up to 24% when passing from Potsdam domain to Vaihingen domain. This margin can be increased by addition of other labeled data from the target domain. To minimize the cost of supervision in the translation process, we proposed a methodology to use these labeled data efficiently.

scholarly journals Unsupervised Domain Adaptation Using Generative Adversarial Networks for Semantic Segmentation of Aerial Images

2019 ◽  
Vol 11 (11) ◽  
pp. 1369 ◽  
Author(s):  
Bilel Benjdira ◽  
Yakoub Bazi ◽  
Anis Koubaa ◽  
Kais Ouni
Keyword(s):  
Urban Areas ◽  
Traffic Management ◽  
Domain Adaptation ◽  
Semantic Segmentation ◽  
Aerial Images ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Adversarial Networks ◽  
Segmentation Accuracy ◽  
New City

Segmenting aerial images is of great potential in surveillance and scene understanding of urban areas. It provides a mean for automatic reporting of the different events that happen in inhabited areas. This remarkably promotes public safety and traffic management applications. After the wide adoption of convolutional neural networks methods, the accuracy of semantic segmentation algorithms could easily surpass 80% if a robust dataset is provided. Despite this success, the deployment of a pretrained segmentation model to survey a new city that is not included in the training set significantly decreases accuracy. This is due to the domain shift between the source dataset on which the model is trained and the new target domain of the new city images. In this paper, we address this issue and consider the challenge of domain adaptation in semantic segmentation of aerial images. We designed an algorithm that reduces the domain shift impact using generative adversarial networks (GANs). In the experiments, we tested the proposed methodology on the International Society for Photogrammetry and Remote Sensing (ISPRS) semantic segmentation dataset and found that our method improves overall accuracy from 35% to 52% when passing from the Potsdam domain (considered as source domain) to the Vaihingen domain (considered as target domain). In addition, the method allows efficiently recovering the inverted classes due to sensor variation. In particular, it improves the average segmentation accuracy of the inverted classes due to sensor variation from 14% to 61%.

scholarly journals TriGAN: image-to-image translation for multi-source domain adaptation

2021 ◽  
Vol 32 (1) ◽  
Author(s):  
Subhankar Roy ◽  
Aliaksandr Siarohin ◽  
Enver Sangineto ◽  
Nicu Sebe ◽  
Elisa Ricci
Keyword(s):  
Domain Adaptation ◽  
Image Features ◽  
Generative Adversarial Networks ◽  
Source Image ◽  
Multiple Sources ◽  
Target Domain ◽  
Invariant Representation ◽  
Source Domain ◽  
Practical Applications ◽  
Adversarial Networks

AbstractMost domain adaptation methods consider the problem of transferring knowledge to the target domain from a single-source dataset. However, in practical applications, we typically have access to multiple sources. In this paper we propose the first approach for multi-source domain adaptation (MSDA) based on generative adversarial networks. Our method is inspired by the observation that the appearance of a given image depends on three factors: the domain, the style (characterized in terms of low-level features variations) and the content. For this reason, we propose to project the source image features onto a space where only the dependence from the content is kept, and then re-project this invariant representation onto the pixel space using the target domain and style. In this way, new labeled images can be generated which are used to train a final target classifier. We test our approach using common MSDA benchmarks, showing that it outperforms state-of-the-art methods.

scholarly journals Domain Adaptation for Semantic Segmentation of Historical Panchromatic Orthomosaics in Central Africa

2021 ◽  
Vol 10 (8) ◽  
pp. 523
Author(s):  
Nicholus Mboga ◽  
Stefano D’Aronco ◽  
Tais Grippa ◽  
Charlotte Pelletier ◽  
Stefanos Georganos ◽  
...  
Keyword(s):  
Land Cover ◽  
Domain Adaptation ◽  
Central Africa ◽  
Semantic Segmentation ◽  
Target Domain ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
The Cost ◽  
Performance Gains

Multitemporal environmental and urban studies are essential to guide policy making to ultimately improve human wellbeing in the Global South. Land-cover products derived from historical aerial orthomosaics acquired decades ago can provide important evidence to inform long-term studies. To reduce the manual labelling effort by human experts and to scale to large, meaningful regions, we investigate in this study how domain adaptation techniques and deep learning can help to efficiently map land cover in Central Africa. We propose and evaluate a methodology that is based on unsupervised adaptation to reduce the cost of generating reference data for several cities and across different dates. We present the first application of domain adaptation based on fully convolutional networks for semantic segmentation of a dataset of historical panchromatic orthomosaics for land-cover generation for two focus cities Goma-Gisenyi and Bukavu. Our experimental evaluation shows that the domain adaptation methods can reach an overall accuracy between 60% and 70% for different regions. If we add a small amount of labelled data from the target domain, too, further performance gains can be achieved.

scholarly journals Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation

2019 ◽  
Vol 33 ◽  
pp. 5581-5588 ◽  
Author(s):  
Yonghao Xu ◽  
Bo Du ◽  
Lefei Zhang ◽  
Qian Zhang ◽  
Guoli Wang ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Great Success ◽  
Learning Models ◽  
Target Domain ◽  
Attention Networks ◽  
Source Domain ◽  
Benchmark Datasets ◽  
Different Levels

Recent years have witnessed the great success of deep learning models in semantic segmentation. Nevertheless, these models may not generalize well to unseen image domains due to the phenomenon of domain shift. Since pixel-level annotations are laborious to collect, developing algorithms which can adapt labeled data from source domain to target domain is of great significance. To this end, we propose self-ensembling attention networks to reduce the domain gap between different datasets. To the best of our knowledge, the proposed method is the first attempt to introduce selfensembling model to domain adaptation for semantic segmentation, which provides a different view on how to learn domain-invariant features. Besides, since different regions in the image usually correspond to different levels of domain gap, we introduce the attention mechanism into the proposed framework to generate attention-aware features, which are further utilized to guide the calculation of consistency loss in the target domain. Experiments on two benchmark datasets demonstrate that the proposed framework can yield competitive performance compared with the state of the art methods.

scholarly journals One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networks

2019 ◽  
Author(s):  
Tao He ◽  
Yuan-Fang Li ◽  
Lianli Gao ◽  
Dongxiang Zhang ◽  
Jingkuan Song
Keyword(s):  
Adaptive Learning ◽  
Digital Data ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Source Domain ◽  
Learning Framework ◽  
Common Space ◽  
Adversarial Networks ◽  
Shared Space ◽  
Benchmark Datasets

With the recent explosive increase of digital data, image recognition and retrieval become a critical practical application. Hashing is an effective solution to this problem, due to its low storage requirement and high query speed. However, most of past works focus on hashing in a single (source) domain. Thus, the learned hash function may not adapt well in a new (target) domain that has a large distributional difference with the source domain. In this paper, we explore an end-to-end domain adaptive learning framework that simultaneously and precisely generates discriminative hash codes and classifies target domain images. Our method encodes two domains images into a semantic common space, followed by two independent generative adversarial networks arming at crosswise reconstructing two domains’ images, reducing domain disparity and improving alignment in the shared space. We evaluate our framework on four public benchmark datasets, all of which show that our method is superior to the other state-of-the-art methods on the tasks of object recognition and image retrieval.

scholarly journals Exploiting Images for Video Recognition with Hierarchical Generative Adversarial Networks

2018 ◽  
Author(s):  
Feiwu Yu ◽  
Xinxiao Wu ◽  
Yuchao Sun ◽  
Lixin Duan
Keyword(s):  
Domain Adaptation ◽  
Feature Representation ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Feature Representations ◽  
Adversarial Networks ◽  
Video Recognition ◽  
Novel Method ◽  
High Level ◽  
Heterogeneous Feature

Existing deep learning methods of video recognition usually require a large number of labeled videos for training. But for a new task, videos are often unlabeled and it is also time-consuming and labor-intensive to annotate them. Instead of human annotation, we try to make use of existing fully labeled images to help recognize those videos. However, due to the problem of domain shifts and heterogeneous feature representations, the performance of classifiers trained on images may be dramatically degraded for video recognition tasks. In this paper, we propose a novel method, called Hierarchical Generative Adversarial Networks (HiGAN), to enhance recognition in videos (i.e., target domain) by transferring knowledge from images (i.e., source domain). The HiGAN model consists of a \emph{low-level} conditional GAN and a \emph{high-level} conditional GAN. By taking advantage of these two-level adversarial learning, our method is capable of learning a domain-invariant feature representation of source images and target videos. Comprehensive experiments on two challenging video recognition datasets (i.e. UCF101 and HMDB51) demonstrate the effectiveness of the proposed method when compared with the existing state-of-the-art domain adaptation methods.

scholarly journals Unsupervised Domain Expansion for Visual Categorization

2021 ◽  
Vol 17 (4) ◽  
pp. 1-24
Author(s):  
Jie Wang ◽  
Kaibin Tian ◽  
Dayong Ding ◽  
Gang Yang ◽  
Xirong Li
Keyword(s):  
Domain Adaptation ◽  
Unlabeled Data ◽  
Target Domain ◽  
Source Domain ◽  
Visual Categorization ◽  
Performance Loss ◽  
Deep Model ◽  
Knowledge Distillation ◽  
The Cost

Expanding visual categorization into a novel domain without the need of extra annotation has been a long-term interest for multimedia intelligence. Previously, this challenge has been approached by unsupervised domain adaptation (UDA). Given labeled data from a source domain and unlabeled data from a target domain, UDA seeks for a deep representation that is both discriminative and domain-invariant. While UDA focuses on the target domain, we argue that the performance on both source and target domains matters, as in practice which domain a test example comes from is unknown. In this article, we extend UDA by proposing a new task called unsupervised domain expansion (UDE), which aims to adapt a deep model for the target domain with its unlabeled data, meanwhile maintaining the model’s performance on the source domain. We propose Knowledge Distillation Domain Expansion (KDDE) as a general method for the UDE task. Its domain-adaptation module can be instantiated with any existing model. We develop a knowledge distillation-based learning mechanism, enabling KDDE to optimize a single objective wherein the source and target domains are equally treated. Extensive experiments on two major benchmarks, i.e., Office-Home and DomainNet, show that KDDE compares favorably against four competitive baselines, i.e., DDC, DANN, DAAN, and CDAN, for both UDA and UDE tasks. Our study also reveals that the current UDA models improve their performance on the target domain at the cost of noticeable performance loss on the source domain.

scholarly journals From shallow to deep: exploiting feature-based classifiers for domain adaptation in semantic segmentation

2021 ◽  
Author(s):  
Alex Matskevych ◽  
Adrian Wolny ◽  
Constantin Pape ◽  
Anna Kreshuk
Keyword(s):  
Random Forest ◽  
Domain Adaptation ◽  
Semantic Segmentation ◽  
Training Data ◽  
Target Domain ◽  
Segmentation Accuracy ◽  
Challenging Tasks ◽  
Feature Based ◽  
The Cost ◽  
Nuclear Segmentation

The remarkable performance of Convolutional Neural Networks on image segmentation tasks comes at the cost of a large amount of pixelwise annotated images that have to be segmented for training. In contrast, feature-based learning methods, such as the Random Forest, require little training data, but never reach the segmentation accuracy of CNNs. This work bridges the two approaches in a transfer learning setting. We show that a CNN can be trained to correct the errors of the Random Forest in the source domain and then be applied to correct such errors in the target domain without retraining, as the domain shift between the Random Forest predictions is much smaller than between the raw data. By leveraging a few brushstrokes as annotations in the target domain, the method can deliver segmentations that are sufficiently accurate to act as pseudo-labels for target-domain CNN training. We demonstrate the performance of the method on several datasets with the challenging tasks of mitochondria, membrane and nuclear segmentation. It yields excellent performance compared to microscopy domain adaptation baselines, especially when a significant domain shift is involved.

Sign in / Sign up

Export Citation Format

Share Document