scholarly journals Generative Learning for Postprocessing Semantic Segmentation Predictions: A Lightweight Conditional Generative Adversarial Network Based on Pix2pix to Improve the Extraction of Road Surface Areas

Land ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 79
Author(s):  
Calimanut-Ionut Cira ◽  
Miguel-Ángel Manso-Callejo ◽  
Ramón Alcarria ◽  
Teresa Fernández Pareja ◽  
Borja Bordel Sánchez ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Road Surface ◽  
Complex Nature ◽  
Generative Learning ◽  
Generative Adversarial Network ◽  
Maximum Increase ◽  
Qualitative Comparison ◽  
The Road ◽  
Surface Areas ◽  
Adversarial Network

Remote sensing experts have been actively using deep neural networks to solve extraction tasks in high-resolution aerial imagery by means of supervised semantic segmentation operations. However, the extraction operation is imperfect, due to the complex nature of geospatial objects, limitations of sensing resolution, or occlusions present in the scenes. In this work, we tackle the challenge of postprocessing semantic segmentation predictions of road surface areas obtained with a state-of-the-art segmentation model and present a technique based on generative learning and image-to-image translations concepts to improve these initial segmentation predictions. The proposed model is a conditional Generative Adversarial Network based on Pix2pix, heavily modified for computational efficiency (92.4% decrease in the number of parameters in the generator network and 61.3% decrease in the discriminator network). The model is trained to learn the distribution of the road network present in official cartography, using a novel dataset containing 6784 tiles of 256 × 256 pixels in size, covering representative areas of Spain. Afterwards, we conduct a metrical comparison using the Intersection over Union (IoU) score (measuring the ratio between the overlap and union areas) on a novel testing set containing 1696 tiles (unseen during training) and observe a maximum increase of 11.6% in the IoU score (from 0.6726 to 0.7515). In the end, we conduct a qualitative comparison to visually assess the effectiveness of the technique and observe great improvements with respect to the initial semantic segmentation predictions.

scholarly journals Improving Road Surface Area Extraction via Semantic Segmentation with Conditional Generative Learning for Deep Inpainting Operations

2022 ◽  
Vol 11 (1) ◽  
pp. 43
Author(s):  
Calimanut-Ionut Cira ◽  
Martin Kada ◽  
Miguel-Ángel Manso-Callejo ◽  
Ramón Alcarria ◽  
Borja Bordel Bordel Sanchez
Keyword(s):  
Surface Area ◽  
Large Scale ◽  
Semantic Segmentation ◽  
Learning Task ◽  
Road Surface ◽  
Generative Learning ◽  
Post Processing ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Unseen Data

The road surface area extraction task is generally carried out via semantic segmentation over remotely-sensed imagery. However, this supervised learning task is often costly as it requires remote sensing images labelled at the pixel level, and the results are not always satisfactory (presence of discontinuities, overlooked connection points, or isolated road segments). On the other hand, unsupervised learning does not require labelled data and can be employed for post-processing the geometries of geospatial objects extracted via semantic segmentation. In this work, we implement a conditional Generative Adversarial Network to reconstruct road geometries via deep inpainting procedures on a new dataset containing unlabelled road samples from challenging areas present in official cartographic support from Spain. The goal is to improve the initial road representations obtained with semantic segmentation models via generative learning. The performance of the model was evaluated on unseen data by conducting a metrical comparison where a maximum Intersection over Union (IoU) score improvement of 1.3% was observed when compared to the initial semantic segmentation result. Next, we evaluated the appropriateness of applying unsupervised generative learning using a qualitative perceptual validation to identify the strengths and weaknesses of the proposed method in very complex scenarios and gain a better intuition of the model’s behaviour when performing large-scale post-processing with generative learning and deep inpainting procedures and observed important improvements in the generated data.

scholarly journals CNN-Based Road-Surface Crack Detection Model That Responds to Brightness Changes

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1402
Author(s):  
Taehee Lee ◽  
Yeohwan Yoon ◽  
Chanjun Chun ◽  
Seungki Ryu
Keyword(s):  
Surface Crack ◽  
Autonomous Vehicles ◽  
Crack Detection ◽  
Model Performance ◽  
Semantic Segmentation ◽  
Road Surface ◽  
Image Brightness ◽  
The Road ◽  
Detection Model ◽  
Artificial Intelligence Models

Poor road-surface conditions pose a significant safety risk to vehicle operation, especially in the case of autonomous vehicles. Hence, maintenance of road surfaces will become even more important in the future. With the development of deep learning-based computer image processing technology, artificial intelligence models that evaluate road conditions are being actively researched. However, as the lighting conditions of the road surface vary depending on the weather, the model performance may degrade for an image whose brightness falls outside the range of the learned image, even for the same road. In this study, a semantic segmentation model with an autoencoder structure was developed for detecting road surface along with a CNN-based image preprocessing model. This setup ensures better road-surface crack detection by adjusting the image brightness before it is input into the road-crack detection model. When the preprocessing model was applied, the road-crack segmentation model exhibited consistent performance even under varying brightness values.

scholarly journals Improving Road Semantic Segmentation Using Generative Adversarial Network

IEEE Access ◽  
2021 ◽  
Vol 9 ◽  
pp. 64381-64392
Author(s):  
Abolfazl Abdollahi ◽  
Biswajeet Pradhan ◽  
Gaurav Sharma ◽  
Khairul Nizam Abdul Maulud ◽  
Abdullah Alamri
Keyword(s):  
Semantic Segmentation ◽  
Generative Adversarial Network ◽  
Adversarial Network

FISS GAN: A Generative Adversarial Network for Foggy Image Semantic Segmentation

2021 ◽  
Vol 8 (8) ◽  
pp. 1428-1439
Author(s):  
Kunhua Liu ◽  
Zihao Ye ◽  
Hongyan Guo ◽  
Dongpu Cao ◽  
Long Chen ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Generative Adversarial Network ◽  
Adversarial Network

scholarly journals Semantic Segmentation of White Matter in FDG-PET Using Generative Adversarial Network

2020 ◽  
Vol 33 (4) ◽  
pp. 816-825
Author(s):  
Kyeong Taek Oh ◽  
Sangwon Lee ◽  
Haeun Lee ◽  
Mijin Yun ◽  
Sun K. Yoo
Keyword(s):  
White Matter ◽  
Fdg Pet ◽  
Semantic Segmentation ◽  
Generative Adversarial Network ◽  
Adversarial Network

scholarly journals Separating Chinese Character from Noisy Background Using GAN

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Bin Huang ◽  
Jiaqi Lin ◽  
Jinming Liu ◽  
Jie Chen ◽  
Jiemin Zhang ◽  
...  
Keyword(s):  
Character Recognition ◽  
Optical Character Recognition ◽  
Complex Structure ◽  
Real Data ◽  
Semantic Segmentation ◽  
Chinese Characters ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Basic Network ◽  
Noisy Background

Separating printed or handwritten characters from a noisy background is valuable for many applications including test paper autoscoring. The complex structure of Chinese characters makes it difficult to obtain the goal because of easy loss of fine details and overall structure in reconstructed characters. This paper proposes a method for separating Chinese characters based on generative adversarial network (GAN). We used ESRGAN as the basic network structure and applied dilated convolution and a novel loss function that improve the quality of reconstructed characters. Four popular Chinese fonts (Hei, Song, Kai, and Imitation Song) on real data collection were tested, and the proposed design was compared with other semantic segmentation approaches. The experimental results showed that the proposed method effectively separates Chinese characters from noisy background. In particular, our methods achieve better results in terms of Intersection over Union (IoU) and optical character recognition (OCR) accuracy.

scholarly journals Semantic Segmentation of Remote Sensing Image Based on GAN and FCN Network Model

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Liang Tian ◽  
Xiaorou Zhong ◽  
Ming Chen
Keyword(s):  
Remote Sensing ◽  
High Efficiency ◽  
Semantic Segmentation ◽  
Remote Sensing Image ◽  
Image Feature ◽  
Generative Adversarial Network ◽  
Current Image ◽  
Adversarial Network ◽  
Complex Image ◽  
Segmentation Methods

Accurate remote sensing image segmentation can guide human activities well, but current image semantic segmentation methods cannot meet the high-precision semantic recognition requirements of complex images. In order to further improve the accuracy of remote sensing image semantic segmentation, this paper proposes a new image semantic segmentation method based on Generative Adversarial Network (GAN) and Fully Convolutional Neural Network (FCN). This method constructs a deep semantic segmentation network based on FCN, which can enhance the receptive field of the model. GAN is integrated into FCN semantic segmentation network to synthesize the global image feature information and then accurately segment the complex remote sensing image. Through experiments on a variety of datasets, it can be seen that the proposed method can meet the high-efficiency requirements of complex image semantic segmentation and has good semantic segmentation capabilities.

scholarly journals A method for improving semantic segmentation using thermographic images in infants

2022 ◽  
Vol 22 (1) ◽  
Author(s):  
Hidetsugu Asano ◽  
Eiji Hirakawa ◽  
Hayato Hayashi ◽  
Keisuke Hamada ◽  
Yuto Asayama ◽  
...  
Keyword(s):  
Temperature Distribution ◽  
Semantic Segmentation ◽  
The Body ◽  
Whole Body ◽  
Clinical Settings ◽  
Generative Adversarial Network ◽  
Thermal Images ◽  
Adversarial Network ◽  
Manual Selection ◽  
Peripheral Temperature

Abstract Background Regulation of temperature is clinically important in the care of neonates because it has a significant impact on prognosis. Although probes that make contact with the skin are widely used to monitor temperature and provide spot central and peripheral temperature information, they do not provide details of the temperature distribution around the body. Although it is possible to obtain detailed temperature distributions using multiple probes, this is not clinically practical. Thermographic techniques have been reported for measurement of temperature distribution in infants. However, as these methods require manual selection of the regions of interest (ROIs), they are not suitable for introduction into clinical settings in hospitals. Here, we describe a method for segmentation of thermal images that enables continuous quantitative contactless monitoring of the temperature distribution over the whole body of neonates. Methods The semantic segmentation method, U-Net, was applied to thermal images of infants. The optimal combination of Weight Normalization, Group Normalization, and Flexible Rectified Linear Unit (FReLU) was evaluated. U-Net Generative Adversarial Network (U-Net GAN) was applied to thermal images, and a Self-Attention (SA) module was finally applied to U-Net GAN (U-Net GAN + SA) to improve precision. The semantic segmentation performance of these methods was evaluated. Results The optimal semantic segmentation performance was obtained with application of FReLU and Group Normalization to U-Net, showing accuracy of 92.9% and Mean Intersection over Union (mIoU) of 64.5%. U-Net GAN improved the performance, yielding accuracy of 93.3% and mIoU of 66.9%, and U-Net GAN + SA showed further improvement with accuracy of 93.5% and mIoU of 70.4%. Conclusions FReLU and Group Normalization are appropriate semantic segmentation methods for application to neonatal thermal images. U-Net GAN and U-Net GAN + SA significantly improved the mIoU of segmentation.

Sign in / Sign up

Export Citation Format

Share Document