Semantic segmentation network combined with edge detection for building extraction in remote sensing images

Deep learning models have brought great breakthroughs in building extraction from high-resolution optical remote-sensing images. Among recent research, the self-attention module has called up a storm in many fields, including building extraction. However, most current deep learning models loading with the self-attention module still lose sight of the reconstruction bias’s effectiveness. Through tipping the balance between the abilities of encoding and decoding, i.e., making the decoding network be much more complex than the encoding network, the semantic segmentation ability will be reinforced. To remedy the research weakness in combing self-attention and reconstruction-bias modules for building extraction, this paper presents a U-Net architecture that combines self-attention and reconstruction-bias modules. In the encoding part, a self-attention module is added to learn the attention weights of the inputs. Through the self-attention module, the network will pay more attention to positions where there may be salient regions. In the decoding part, multiple large convolutional up-sampling operations are used for increasing the reconstruction ability. We test our model on two open available datasets: the WHU and Massachusetts Building datasets. We achieve IoU scores of 89.39% and 73.49% for the WHU and Massachusetts Building datasets, respectively. Compared with several recently famous semantic segmentation methods and representative building extraction methods, our method’s results are satisfactory.

Download Full-text

ERN: Edge Loss Reinforced Semantic Segmentation Network for Remote Sensing Images

Remote Sensing ◽

10.3390/rs10091339 ◽

2018 ◽

Vol 10 (9) ◽

pp. 1339 ◽

Cited By ~ 16

Author(s):

Shuo Liu ◽

Wenrui Ding ◽

Chunhui Liu ◽

Yu Liu ◽

Yufeng Wang ◽

...

Keyword(s):

Remote Sensing ◽

Edge Detection ◽

Qualitative Evaluation ◽

Semantic Segmentation ◽

Model Complexity ◽

Semantic Ambiguity ◽

Remote Sensing Images ◽

Future Networks ◽

Spatial Boundary ◽

Boundary Information

The semantic segmentation of remote sensing images faces two major challenges: high inter-class similarity and interference from ubiquitous shadows. In order to address these issues, we develop a novel edge loss reinforced semantic segmentation network (ERN) that leverages the spatial boundary context to reduce the semantic ambiguity. The main contributions of this paper are as follows: (1) we propose a novel end-to-end semantic segmentation network for remote sensing, which involves multiple weighted edge supervisions to retain spatial boundary information; (2) the main representations of the network are shared between the edge loss reinforced structures and semantic segmentation, which means that the ERN simultaneously achieves semantic segmentation and edge detection without significantly increasing the model complexity; and (3) we explore and discuss different ERN schemes to guide the design of future networks. Extensive experimental results on two remote sensing datasets demonstrate the effectiveness of our approach both in quantitative and qualitative evaluation. Specifically, the semantic segmentation performance in shadow-affected regions is significantly improved.

Download Full-text

Combined Multi-Layer Feature Fusion and Edge Detection Method for Distributed Photovoltaic Power Station Identification

Energies ◽

10.3390/en13246742 ◽

2020 ◽

Vol 13 (24) ◽

pp. 6742

Author(s):

Yongshi Jie ◽

Xianhua Ji ◽

Anzhi Yue ◽

Jingbo Chen ◽

Yupeng Deng ◽

...

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Edge Detection ◽

Semantic Segmentation ◽

Remote Sensing Images ◽

Power Station ◽

Data Set ◽

Photovoltaic Panels ◽

Power Stations ◽

Photovoltaic Power

Distributed photovoltaic power stations are an effective way to develop and utilize solar energy resources. Using high-resolution remote sensing images to obtain the locations, distribution, and areas of distributed photovoltaic power stations over a large region is important to energy companies, government departments, and investors. In this paper, a deep convolutional neural network was used to extract distributed photovoltaic power stations from high-resolution remote sensing images automatically, accurately, and efficiently. Based on a semantic segmentation model with an encoder-decoder structure, a gated fusion module was introduced to address the problem that small photovoltaic panels are difficult to identify. Further, to solve the problems of blurred edges in the segmentation results and that adjacent photovoltaic panels can easily be adhered, this work combines an edge detection network and a semantic segmentation network for multi-task learning to extract the boundaries of photovoltaic panels in a refined manner. Comparative experiments conducted on the Duke California Solar Array data set and a self-constructed Shanghai Distributed Photovoltaic Power Station data set show that, compared with SegNet, LinkNet, UNet, and FPN, the proposed method obtained the highest identification accuracy on both data sets, and its F1-scores reached 84.79% and 94.03%, respectively. These results indicate that effectively combining multi-layer features with a gated fusion module and introducing an edge detection network to refine the segmentation improves the accuracy of distributed photovoltaic power station identification.

Download Full-text

Precise Extraction of Buildings from High-Resolution Remote-Sensing Images Based on Semantic Edges and Segmentation

Remote Sensing ◽

10.3390/rs13163083 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3083

Author(s):

Liegang Xia ◽

Junxia Zhang ◽

Xiongbo Zhang ◽

Haiping Yang ◽

Meixia Xu

Keyword(s):

Remote Sensing ◽

Edge Detection ◽

Semantic Segmentation ◽

Building Extraction ◽

The Past ◽

Basic Task ◽

Effective Combination ◽

Level Information ◽

Drawing Tasks ◽

High Level

Building extraction is a basic task in the field of remote sensing, and it has also been a popular research topic in the past decade. However, the shape of the semantic polygon generated by semantic segmentation is irregular and does not match the actual building boundary. The boundary of buildings generated by semantic edge detection has difficulty ensuring continuity and integrity. Due to the aforementioned problems, we cannot directly apply the results in many drawing tasks and engineering applications. In this paper, we propose a novel convolutional neural network (CNN) model based on multitask learning, Dense D-LinkNet (DDLNet), which adopts full-scale skip connections and edge guidance module to ensure the effective combination of low-level information and high-level information. DDLNet has good adaptability to both semantic segmentation tasks and edge detection tasks. Moreover, we propose a universal postprocessing method that integrates semantic edges and semantic polygons. It can solve the aforementioned problems and more accurately locate buildings, especially building boundaries. The experimental results show that DDLNet achieves great improvements compared with other edge detection and semantic segmentation networks. Our postprocessing method is effective and universal.

Download Full-text

Building Extraction from Very-High-Resolution Remote Sensing Images Using Semi-Supervised Semantic Edge Detection

Remote Sensing ◽

10.3390/rs13112187 ◽

2021 ◽

Vol 13 (11) ◽

pp. 2187

Author(s):

Liegang Xia ◽

Xiongbo Zhang ◽

Junxia Zhang ◽

Haiping Yang ◽

Tingting Chen

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

High Resolution ◽

Edge Detection ◽

Network Architecture ◽

Semantic Segmentation ◽

Google Earth ◽

Remote Sensing Images ◽

Building Roof ◽

Social Applications

The automated detection of buildings in remote sensing images enables understanding the distribution information of buildings, which is indispensable for many geographic and social applications, such as urban planning, change monitoring and population estimation. The performance of deep learning in images often depends on a large number of manually labeled samples, the production of which is time-consuming and expensive. Thus, this study focuses on reducing the number of labeled samples used and proposing a semi-supervised deep learning approach based on an edge detection network (SDLED), which is the first to introduce semi-supervised learning to the edge detection neural network for extracting building roof boundaries from high-resolution remote sensing images. This approach uses a small number of labeled samples and abundant unlabeled images for joint training. An expert-level semantic edge segmentation model is trained based on labeled samples, which guides unlabeled images to generate pseudo-labels automatically. The inaccurate label sets and manually labeled samples are used to update the semantic edge model together. Particularly, we modified the semantic segmentation network D-LinkNet to obtain high-quality pseudo-labels. Specifically, the main network architecture of D-LinkNet is retained while the multi-scale fusion is added in its second half to improve its performance on edge detection. The SDLED was tested on high-spatial-resolution remote sensing images taken from Google Earth. Results show that the SDLED performs better than the fully supervised method. Moreover, when the trained models were used to predict buildings in the neighboring counties, our approach was superior to the supervised way, with line IoU improvement of at least 6.47% and F1 score improvement of at least 7.49%.

Download Full-text

Semantic Segmentation for Buildings of Large Intra-Class Variation in Remote Sensing Images with O-GAN

Remote Sensing ◽

10.3390/rs13030475 ◽

2021 ◽

Vol 13 (3) ◽

pp. 475

Author(s):

Shuting Sun ◽

Lin Mu ◽

Lizhe Wang ◽

Peng Liu ◽

Xiaolei Liu ◽

...

Keyword(s):

Remote Sensing ◽

Economic Status ◽

Back Propagation ◽

Tall Buildings ◽

Semantic Segmentation ◽

Generative Adversarial Networks ◽

Building Extraction ◽

Remote Sensing Images ◽

Feature Vectors ◽

Classification Feature

Remote sensing building extraction is of great importance to many applications, such as urban planning and economic status assessment. Deep learning with deep network structures and back-propagation optimization can automatically learn features of targets in high-resolution remote sensing images. However, it is also obvious that the generalizability of deep networks is almost entirely dependent on the quality and quantity of the labels. Therefore, building extraction performances will be greatly affected if there is a large intra-class variation among samples of one class target. To solve the problem, a subdivision method for reducing intra-class differences is proposed to enhance semantic segmentation. We proposed that backgrounds and targets be separately generated by two orthogonal generative adversarial networks (O-GAN). The two O-GANs are connected by adding the new loss function to their discriminators. To better extract building features, drawing on the idea of fine-grained image classification, feature vectors for a target are obtained through an intermediate convolution layer of O-GAN with selective convolutional descriptor aggregation (SCDA). Subsequently, feature vectors are clustered into new, different subdivisions to train semantic segmentation networks. In the prediction stages, the subdivisions will be merged into one class. Experiments were conducted with remote sensing images of the Tibet area, where there are both tall buildings and herdsmen’s tents. The results indicate that, compared with direct semantic segmentation, the proposed subdivision method can make an improvement on accuracy of about 4%. Besides, statistics and visualizing building features validated the rationality of features and subdivisions.

Download Full-text

Improved Anchor-Free Instance Segmentation for Building Extraction from High-Resolution Remote Sensing Images

Remote Sensing ◽

10.3390/rs12182910 ◽

2020 ◽

Vol 12 (18) ◽

pp. 2910

Author(s):

Tong Wu ◽

Yuan Hu ◽

Ling Peng ◽

Ruonan Chen

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Semantic Segmentation ◽

Aerial Images ◽

Building Extraction ◽

Remote Sensing Images ◽

Highly Sensitive ◽

Segmentation Methods ◽

Speed And Accuracy ◽

Instance Segmentation

Building extraction from high-resolution remote sensing images plays a vital part in urban planning, safety supervision, geographic databases updates, and some other applications. Several researches are devoted to using convolutional neural network (CNN) to extract buildings from high-resolution satellite/aerial images. There are two major methods, one is the CNN-based semantic segmentation methods, which can not distinguish different objects of the same category and may lead to edge connection. The other one is CNN-based instance segmentation methods, which rely heavily on pre-defined anchors, and result in the highly sensitive, high computation/storage cost and imbalance between positive and negative samples. Therefore, in this paper, we propose an improved anchor-free instance segmentation method based on CenterMask with spatial and channel attention-guided mechanisms and improved effective backbone network for accurate extraction of buildings in high-resolution remote sensing images. Then we analyze the influence of different parameters and network structure on the performance of the model, and compare the performance for building extraction of Mask R-CNN, Mask Scoring R-CNN, CenterMask, and the improved CenterMask in this paper. Experimental results show that our improved CenterMask method can successfully well-balanced performance in terms of speed and accuracy, which achieves state-of-the-art performance at real-time speed.

Download Full-text

Building Extraction and Number Statistics in WUI Areas Based on UNet Structure and Ensemble Learning

Remote Sensing ◽

10.3390/rs13061172 ◽

2021 ◽

Vol 13 (6) ◽

pp. 1172

Author(s):

De-Yue Chen ◽

Ling Peng ◽

Wei-Chao Li ◽

Yin-Da Wang

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

High Resolution ◽

Ensemble Learning ◽

Semantic Segmentation ◽

Regional Governance ◽

Building Extraction ◽

Contour Extraction ◽

Remote Sensing Images ◽

Building Information

Following the advancement and progression of urbanization, management problems of the wildland–urban interface (WUI) have become increasingly serious. WUI regional governance issues involve many factors including climate, humanities, etc., and have attracted attention and research from all walks of life. Building research plays a vital part in the WUI area. Building location is closely related with the planning and management of the WUI area, and the number of buildings is related to the rescue arrangement. There are two major methods to obtain this building information: one is to obtain them from relevant agencies, which is slow and lacks timeliness, while the other approach is to extract them from high-resolution remote sensing images, which is relatively inexpensive and offers improved timeliness. Inspired by the recent successful application of deep learning, in this paper, we propose a method for extracting building information from high-resolution remote sensing images based on deep learning, which is combined with ensemble learning to extract the building location. Further, we use the idea of image anomaly detection to estimate the number of buildings. After verification on two datasets, we obtain superior semantic segmentation results and achieve better building contour extraction and number estimation.

Download Full-text

Semantic Segmentation of Remote Sensing Images With Sparse Annotations

IEEE Geoscience and Remote Sensing Letters ◽

10.1109/lgrs.2021.3051053 ◽

2021 ◽

pp. 1-5

Author(s):

Yuansheng Hua ◽

Diego Marcos ◽

Lichao Mou ◽

Xiao Xiang Zhu ◽

Devis Tuia

Keyword(s):

Remote Sensing ◽

Semantic Segmentation ◽

Remote Sensing Images

Download Full-text

Convolutional Neural Network for the Semantic Segmentation of Remote Sensing Images

Mobile Networks and Applications ◽

10.1007/s11036-020-01703-3 ◽

2021 ◽

Vol 26 (1) ◽

pp. 200-215

Author(s):

Muhammad Alam ◽

Jian-Feng Wang ◽

Cong Guangpei ◽

LV Yunrong ◽

Yuanfang Chen

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Neural Networks ◽

Image Processing ◽

Deep Learning ◽

Semantic Segmentation ◽

Natural Scene ◽

Remote Sensing Images ◽

Advantages And Disadvantages ◽

Target Segmentation

AbstractIn recent years, the success of deep learning in natural scene image processing boosted its application in the analysis of remote sensing images. In this paper, we applied Convolutional Neural Networks (CNN) on the semantic segmentation of remote sensing images. We improve the Encoder- Decoder CNN structure SegNet with index pooling and U-net to make them suitable for multi-targets semantic segmentation of remote sensing images. The results show that these two models have their own advantages and disadvantages on the segmentation of different objects. In addition, we propose an integrated algorithm that integrates these two models. Experimental results show that the presented integrated algorithm can exploite the advantages of both the models for multi-target segmentation and achieve a better segmentation compared to these two models.

Download Full-text