Domain Adaptation for Semantic Segmentation of Aerial Imagery Using Cycle-Consistent Adversarial Networks

Despite the significant advances noted in semantic segmentation of aerial imagery, a considerable limitation is blocking its adoption in real cases. If we test a segmentation model on a new area that is not included in its initial training set, accuracy will decrease remarkably. This is caused by the domain shift between the new targeted domain and the source domain used to train the model. In this paper, we addressed this challenge and proposed a new algorithm that uses Generative Adversarial Networks (GAN) architecture to minimize the domain shift and increase the ability of the model to work on new targeted domains. The proposed GAN architecture contains two GAN networks. The first GAN network converts the chosen image from the target domain into a semantic label. The second GAN network converts this generated semantic label into an image that belongs to the source domain but conserves the semantic map of the target image. This resulting image will be used by the semantic segmentation model to generate a better semantic label of the first chosen image. Our algorithm is tested on the ISPRS semantic segmentation dataset and improved the global accuracy by a margin up to 24% when passing from Potsdam domain to Vaihingen domain. This margin can be increased by addition of other labeled data from the target domain. To minimize the cost of supervision in the translation process, we proposed a methodology to use these labeled data efficiently.

Download Full-text

Unsupervised Domain Adaptation Using Generative Adversarial Networks for Semantic Segmentation of Aerial Images

Remote Sensing ◽

10.3390/rs11111369 ◽

2019 ◽

Vol 11 (11) ◽

pp. 1369 ◽

Cited By ~ 21

Author(s):

Bilel Benjdira ◽

Yakoub Bazi ◽

Anis Koubaa ◽

Kais Ouni

Keyword(s):

Urban Areas ◽

Traffic Management ◽

Domain Adaptation ◽

Semantic Segmentation ◽

Aerial Images ◽

Generative Adversarial Networks ◽

Target Domain ◽

Adversarial Networks ◽

Segmentation Accuracy ◽

New City

Segmenting aerial images is of great potential in surveillance and scene understanding of urban areas. It provides a mean for automatic reporting of the different events that happen in inhabited areas. This remarkably promotes public safety and traffic management applications. After the wide adoption of convolutional neural networks methods, the accuracy of semantic segmentation algorithms could easily surpass 80% if a robust dataset is provided. Despite this success, the deployment of a pretrained segmentation model to survey a new city that is not included in the training set significantly decreases accuracy. This is due to the domain shift between the source dataset on which the model is trained and the new target domain of the new city images. In this paper, we address this issue and consider the challenge of domain adaptation in semantic segmentation of aerial images. We designed an algorithm that reduces the domain shift impact using generative adversarial networks (GANs). In the experiments, we tested the proposed methodology on the International Society for Photogrammetry and Remote Sensing (ISPRS) semantic segmentation dataset and found that our method improves overall accuracy from 35% to 52% when passing from the Potsdam domain (considered as source domain) to the Vaihingen domain (considered as target domain). In addition, the method allows efficiently recovering the inverted classes due to sensor variation. In particular, it improves the average segmentation accuracy of the inverted classes due to sensor variation from 14% to 61%.

Download Full-text

ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks

IEEE Transactions on Geoscience and Remote Sensing ◽

10.1109/tgrs.2020.2980417 ◽

2020 ◽

Vol 58 (10) ◽

pp. 7178-7193 ◽

Cited By ~ 3

Author(s):

Onur Tasar ◽

S. L. Happy ◽

Yuliya Tarabalka ◽

Pierre Alliez

Keyword(s):

Domain Adaptation ◽

Semantic Segmentation ◽

Generative Adversarial Networks ◽

Color Mapping ◽

Unsupervised Domain Adaptation ◽

Adversarial Networks

Download Full-text

Adversarial Data Augmentation on Breast MRI Segmentation

Applied Sciences ◽

10.3390/app11104554 ◽

2021 ◽

Vol 11 (10) ◽

pp. 4554

Author(s):

João F. Teixeira ◽

Mariana Dias ◽

Eva Batista ◽

Joana Costa ◽

Luís F. Teixeira ◽

...

Keyword(s):

Data Augmentation ◽

Medical Image Analysis ◽

Synthetic Data ◽

Breast Mri ◽

Semantic Segmentation ◽

Adversarial Networks ◽

Augmentation Strategy ◽

Magnetic Resonance Imaging Mri ◽

Segmentation Task ◽

The Impact

The scarcity of balanced and annotated datasets has been a recurring problem in medical image analysis. Several researchers have tried to fill this gap employing dataset synthesis with adversarial networks (GANs). Breast magnetic resonance imaging (MRI) provides complex, texture-rich medical images, with the same annotation shortage issues, for which, to the best of our knowledge, no previous work tried synthesizing data. Within this context, our work addresses the problem of synthesizing breast MRI images from corresponding annotations and evaluate the impact of this data augmentation strategy on a semantic segmentation task. We explored variations of image-to-image translation using conditional GANs, namely fitting the generator’s architecture with residual blocks and experimenting with cycle consistency approaches. We studied the impact of these changes on visual verisimilarity and how an U-Net segmentation model is affected by the usage of synthetic data. We achieved sufficiently realistic-looking breast MRI images and maintained a stable segmentation score even when completely replacing the dataset with the synthetic set. Our results were promising, especially when concerning to Pix2PixHD and Residual CycleGAN architectures.

Download Full-text

Domain Adaptation for Semantic Segmentation of Historical Panchromatic Orthomosaics in Central Africa

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10080523 ◽

2021 ◽

Vol 10 (8) ◽

pp. 523

Author(s):

Nicholus Mboga ◽

Stefano D’Aronco ◽

Tais Grippa ◽

Charlotte Pelletier ◽

Stefanos Georganos ◽

...

Keyword(s):

Land Cover ◽

Domain Adaptation ◽

Central Africa ◽

Semantic Segmentation ◽

Target Domain ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

The Cost ◽

Performance Gains

Multitemporal environmental and urban studies are essential to guide policy making to ultimately improve human wellbeing in the Global South. Land-cover products derived from historical aerial orthomosaics acquired decades ago can provide important evidence to inform long-term studies. To reduce the manual labelling effort by human experts and to scale to large, meaningful regions, we investigate in this study how domain adaptation techniques and deep learning can help to efficiently map land cover in Central Africa. We propose and evaluate a methodology that is based on unsupervised adaptation to reduce the cost of generating reference data for several cities and across different dates. We present the first application of domain adaptation based on fully convolutional networks for semantic segmentation of a dataset of historical panchromatic orthomosaics for land-cover generation for two focus cities Goma-Gisenyi and Bukavu. Our experimental evaluation shows that the domain adaptation methods can reach an overall accuracy between 60% and 70% for different regions. If we add a small amount of labelled data from the target domain, too, further performance gains can be achieved.

Download Full-text

Semantic Segmentation of High Resolution Satellite Imagery using Generative Adversarial Networks with Progressive Growing

Remote Sensing Letters ◽

10.1080/2150704x.2021.1895444 ◽

2021 ◽

Vol 12 (5) ◽

pp. 439-448

Author(s):

Edward Collier ◽

Supratik Mukhopadhyay ◽

Kate Duffy ◽

Sangram Ganguly ◽

Geri Madanguit ◽

...

Keyword(s):

High Resolution ◽

Satellite Imagery ◽

Semantic Segmentation ◽

Generative Adversarial Networks ◽

Adversarial Networks ◽

High Resolution Satellite Imagery

Download Full-text

Supervised Domain Adaptation for Automated Semantic Segmentation of the Atrial Cavity

Entropy ◽

10.3390/e23070898 ◽

2021 ◽

Vol 23 (7) ◽

pp. 898

Author(s):

Marta Saiz-Vivó ◽

Adrián Colomer ◽

Carles Fonfría ◽

Luis Martí-Bonmatí ◽

Valery Naranjo

Keyword(s):

High Performance ◽

Computational Models ◽

Domain Adaptation ◽

Semantic Segmentation ◽

Patient Specific ◽

Mr Images ◽

Training Samples ◽

Volumetric Images ◽

Acquisition Costs ◽

Left And Right

Atrial fibrillation (AF) is the most common cardiac arrhythmia. At present, cardiac ablation is the main treatment procedure for AF. To guide and plan this procedure, it is essential for clinicians to obtain patient-specific 3D geometrical models of the atria. For this, there is an interest in automatic image segmentation algorithms, such as deep learning (DL) methods, as opposed to manual segmentation, an error-prone and time-consuming method. However, to optimize DL algorithms, many annotated examples are required, increasing acquisition costs. The aim of this work is to develop automatic and high-performance computational models for left and right atrium (LA and RA) segmentation from a few labelled MRI volumetric images with a 3D Dual U-Net algorithm. For this, a supervised domain adaptation (SDA) method is introduced to infer knowledge from late gadolinium enhanced (LGE) MRI volumetric training samples (80 LA annotated samples) to a network trained with balanced steady-state free precession (bSSFP) MR images of limited number of annotations (19 RA and LA annotated samples). The resulting knowledge-transferred model SDA outperformed the same network trained from scratch in both RA (Dice equals 0.9160) and LA (Dice equals 0.8813) segmentation tasks.

Download Full-text