Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set

Deep learning approaches have been widely used in building automatic extraction tasks and have made great progress in recent years. However, the missing detection and wrong detection causing by spectrum confusion is still a great challenge. The existing fully convolutional networks (FCNs) cannot effectively distinguish whether the feature differences are from one building or the building and its adjacent non-building objects. In order to overcome the limitations, a building multi-feature fusion refined network (BMFR-Net) was presented in this paper to extract buildings accurately and completely. BMFR-Net is based on an encoding and decoding structure, mainly consisting of two parts: the continuous atrous convolution pyramid (CACP) module and the multiscale output fusion constraint (MOFC) structure. The CACP module is positioned at the end of the contracting path and it effectively minimizes the loss of effective information in multiscale feature extraction and fusion by using parallel continuous small-scale atrous convolution. To improve the ability to aggregate semantic information from the context, the MOFC structure performs predictive output at each stage of the expanding path and integrates the results into the network. Furthermore, the multilevel joint weighted loss function effectively updates parameters well away from the output layer, enhancing the learning capacity of the network for low-level abstract features. The experimental results demonstrate that the proposed BMFR-Net outperforms the other five state-of-the-art approaches in both visual interpretation and quantitative evaluation.

Download Full-text

One View Per City for Buildings Segmentation in Remote-Sensing Images via Fully Convolutional Networks: A Proof-of-Concept Study

Sensors ◽

10.3390/s20010141 ◽

2019 ◽

Vol 20 (1) ◽

pp. 141

Author(s):

Jianguang Li ◽

Wen Li ◽

Cong Jin ◽

Lijuan Yang ◽

Hui He

Keyword(s):

Remote Sensing ◽

Image Analysis ◽

Environmental Benefits ◽

Sustainable Urban Development ◽

Aerial Image ◽

Proof Of Concept ◽

Data Set ◽

Concept Study ◽

Convolutional Networks ◽

Fully Convolutional Networks

The segmentation of buildings in remote-sensing (RS) images plays an important role in monitoring landscape changes. Quantification of these changes can be used to balance economic and environmental benefits and most importantly, to support the sustainable urban development. Deep learning has been upgrading the techniques for RS image analysis. However, it requires a large-scale data set for hyper-parameter optimization. To address this issue, the concept of “one view per city” is proposed and it explores the use of one RS image for parameter settings with the purpose of handling the rest images of the same city by the trained model. The proposal of this concept comes from the observation that buildings of a same city in single-source RS images demonstrate similar intensity distributions. To verify the feasibility, a proof-of-concept study is conducted and five fully convolutional networks are evaluated on five cities in the Inria Aerial Image Labeling database. Experimental results suggest that the concept can be explored to decrease the number of images for model training and it enables us to achieve competitive performance in buildings segmentation with decreased time consumption. Based on model optimization and universal image representation, it is full of potential to improve the segmentation performance, to enhance the generalization capacity, and to extend the application of the concept in RS image analysis.

Download Full-text

Using Convolutional Networks and Satellite Imagery to Predict Disease Density in a Developing Country

10.20944/preprints201910.0319.v1 ◽

2019 ◽

Author(s):

Rahman Sanya ◽

Gilbert Maiga ◽

Ernest Mwebaze

Keyword(s):

Infectious Diseases ◽

Developing Country ◽

Satellite Imagery ◽

Spatial Dynamics ◽

Image Data ◽

Urban Housing ◽

Digital Data ◽

Data Set ◽

Convolutional Networks ◽

Multiple Data

Rapid increase in digital data coupled with advances in deep learning algorithms is opening unprecedented opportunities for incorporating multiple data sources for modeling spatial dynamics of human infectious diseases. We used Convolutional Neural Networks (CNN) in conjunction with satellite imagery-based urban housing and socio-economic data to predict disease density in a developing country setting. We explored both single (uni) and multiple input (multimodality) network architectures for this purpose. We achieved maximum test set accuracy of 81.6 per cent using a single input CNN model built with one convolutional layer and trained using housing image data. However, this fairly good performance was biased in favor of specific disease density classes due to an unbalanced data set despite our use of methods to address the problem. These results suggest CNN are promising for modeling spatial dynamics of human infectious diseases, especially in a developing country setting. Urban housing signals extracted from satellite imagery seem suitable for this purpose, under the same context.

Download Full-text

Dark Spot Detection in SAR Images of Oil Spill Using Segnet

Applied Sciences ◽

10.3390/app8122670 ◽

2018 ◽

Vol 8 (12) ◽

pp. 2670 ◽

Cited By ~ 9

Author(s):

Hao Guo ◽

Guo Wei ◽

Jubai An

Keyword(s):

Oil Spill ◽

Semantic Segmentation ◽

Dark Spot ◽

Sar Images ◽

Data Set ◽

High Noise ◽

Oil Spill Detection ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Spot Detection

Damping Bragg scattering from the ocean surface is the basic underlying principle of synthetic aperture radar (SAR) oil slick detection, and they produce dark spots on SAR images. Dark spot detection is the first step in oil spill detection, which affects the accuracy of oil spill detection. However, some natural phenomena (such as waves, ocean currents, and low wind belts, as well as human factors) may change the backscatter intensity on the surface of the sea, resulting in uneven intensity, high noise, and blurred boundaries of oil slicks or lookalikes. In this paper, Segnet is used as a semantic segmentation model to detect dark spots in oil spill areas. The proposed method is applied to a data set of 4200 from five original SAR images of an oil spill. The effectiveness of the method is demonstrated through the comparison with fully convolutional networks (FCN), an initiator of semantic segmentation models, and some other segmentation methods. It is here observed that the proposed method can not only accurately identify the dark spots in SAR images, but also show a higher robustness under high noise and fuzzy boundary conditions.

Download Full-text

Diurnal and nocturnal cloud segmentation of all-sky imager (ASI) images using enhancement fully convolutional networks

Atmospheric Measurement Techniques ◽

10.5194/amt-12-4713-2019 ◽

2019 ◽

Vol 12 (9) ◽

pp. 4713-4724

Author(s):

Chaojun Shi ◽

Yatong Zhou ◽

Bo Qiu ◽

Jingfei He ◽

Mu Ding ◽

...

Keyword(s):

Color Space ◽

Histogram Equalization ◽

Convolutional Network ◽

Data Set ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Saturation Intensity ◽

Hsi Color Space ◽

Rgb Color Space ◽

Sky Imager

Abstract. Cloud segmentation plays a very important role in astronomical observatory site selection. At present, few researchers segment cloud in nocturnal all-sky imager (ASI) images. This paper proposes a new automatic cloud segmentation algorithm that utilizes the advantages of deep-learning fully convolutional networks (FCNs) to segment cloud pixels from diurnal and nocturnal ASI images; it is called the enhancement fully convolutional network (EFCN). Firstly, all the ASI images in the data set from the Key Laboratory of Optical Astronomy at the National Astronomical Observatories of Chinese Academy of Sciences (CAS) are converted from the red–green–blue (RGB) color space to hue saturation intensity (HSI) color space. Secondly, the I channel of the HSI color space is enhanced by histogram equalization. Thirdly, all the ASI images are converted from the HSI color space to RGB color space. Then after 100 000 iterative trainings based on the ASI images in the training set, the optimum associated parameters of the EFCN-8s model are obtained. Finally, we use the trained EFCN-8s to segment the cloud pixels of the ASI image in the test set. In the experiments our proposed EFCN-8s was compared with four other algorithms (OTSU, FCN-8s, EFCN-32s, and EFCN-16s) using four evaluation metrics. Experiments show that the EFCN-8s is much more accurate in cloud segmentation for diurnal and nocturnal ASI images than the other four algorithms.

Download Full-text

Registration of High Resolution Sar and Optical Satellite Imagery Using Fully Convolutional Networks

IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium ◽

10.1109/igarss.2019.8898714 ◽

2019 ◽

Author(s):

Stefan Hoffmann ◽

Clemens-Alexander Brust ◽

Maha Shadaydeh ◽

Joachim Denzler

Keyword(s):

High Resolution ◽

Satellite Imagery ◽

Convolutional Networks ◽

Fully Convolutional Networks

Download Full-text

Off-Grid DOA Estimation Based on Circularly Fully Convolutional Networks (CFCN) Using Space-Frequency Pseudo-Spectrum

Sensors ◽

10.3390/s21082767 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2767

Author(s):

Wenqiong Zhang ◽

Yiwei Huang ◽

Jianfei Tong ◽

Ming Bao ◽

Xiaodong Li

Keyword(s):

Low Frequency ◽

Doa Estimation ◽

Classification Problem ◽

Training Data ◽

Model Parameters ◽

Data Set ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Space Frequency ◽

Multi Classification

Low-frequency multi-source direction-of-arrival (DOA) estimation has been challenging for micro-aperture arrays. Deep learning (DL)-based models have been introduced to this problem. Generally, existing DL-based methods formulate DOA estimation as a multi-label multi-classification problem. However, the accuracy of these methods is limited by the number of grids, and the performance is overly dependent on the training data set. In this paper, we propose an off-grid DL-based DOA estimation. The backbone is based on circularly fully convolutional networks (CFCN), trained by the data set labeled by space-frequency pseudo-spectra, and provides on-grid DOA proposals. Then, the regressor is developed to estimate the precise DOAs according to corresponding proposals and features. In this framework, spatial phase features are extracted by the circular convolution calculation. The improvement in spatial resolution is converted to increasing the dimensionality of features by rotating convolutional networks. This model ensures that the DOA estimations at different sub-bands have the same interpretation ability and effectively reduce network model parameters. The simulation and semi-anechoic chamber experiment results show that CFCN-based DOA is superior to existing methods in terms of generalization ability, resolution, and accuracy.

Download Full-text

Learn to Extract Building Outline from Misaligned Annotation through Nearest Feature Selector

Remote Sensing ◽

10.3390/rs12172722 ◽

2020 ◽

Vol 12 (17) ◽

pp. 2722

Author(s):

Yuxuan Wang ◽

Guangming Wu ◽

Yimin Guo ◽

Yifei Huang ◽

Ryosuke Shibasaki

Keyword(s):

Large Scale ◽

Rapid Development ◽

Semantic Segmentation ◽

Loss Functions ◽

Aerial Image ◽

Building Extraction ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Feature Selector ◽

Segmentation Task

For efficient building outline extraction, many algorithms, including unsupervised or supervised, have been proposed over the past decades. In recent years, due to the rapid development of the convolutional neural networks, especially fully convolutional networks, building extraction is treated as a semantic segmentation task that deals with the extremely biased positive pixels. The state-of-the-art methods, either through direct or indirect approaches, are mainly focused on better network design. The shifts and rotations, which are coarsely presented in manually created annotations, have long been ignored. Due to the limited number of positive samples, the misalignment will significantly reduce the correctness of pixel-to-pixel loss that might lead to a gradient explosion. To overcome this, we propose a nearest feature selector (NFS) to dynamically re-align the prediction and slightly misaligned annotations. The NFS can be seamlessly appended to existing loss functions and prevent misleading by the errors or misalignment of annotations. Experiments on a large scale aerial image dataset with centered buildings and corresponding building outlines indicate that the additional NFS brings higher performance when compared to existing naive loss functions. In the classic L1 loss, the addition of NFS gains increments of 8.8% of f1-score, 8.9% of kappa coefficient, and 9.8% of Jaccard index, respectively.

Download Full-text

Diurnal and nocturnal cloud segmentation of ASI images using enhancement fully convolutional networks

10.5194/amt-2019-93 ◽

2019 ◽

Author(s):

Chaojun Shi ◽

Yatong Zhou ◽

Bo Qiu ◽

Jingfei He ◽

Mu Ding ◽

...

Keyword(s):

Color Space ◽

Histogram Equalization ◽

Data Set ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Saturation Intensity ◽

Hsi Color Space ◽

Rgb Color Space ◽

Optical Astronomy ◽

Academy Of Sciences

Abstract. Cloud segmentation plays a very important role in the astronomical observatory site selection. At present, few researchers segment cloud in the nocturnal All Sky Imager (ASI) images. This paper proposes a new automatic cloud segmentation algorithm which utilizes the advantages of deep learning fully convolutional networks (FCN) to segment cloud pixels from diurnal and nocturnal ASI images, named enhancement fully convolutional networks (EFCN). Firstly, all the ASI images in the data set from the Key Laboratory of Optical Astronomy at the National Astronomical Observatories of Chinese Academy of Sciences (CAS) are converted from the red-green-blue (RGB) color space to hue-saturation-intensity (HSI) color space. Secondly, the channel of the HSI color space is enhanced by the histogram equalization. Thirdly, all the ASI images are converted from the HSI color space to RGB color space. Then after 100000 times iterative training based on the ASI images in the training set, the optimum associated parameters of the EFCN-8s model are obtained. Finally, we use the trained EFCN-8s to segment the cloud pixels of the ASI image in the test set. In the experiments our proposed EFCN-8s was compared with other four algorithms (OTSU, FCN-8s, EFCN-32s, and EFCN-16s) using four evaluation metrics. Experiments show that the EFCN-8s is much more accurate in the cloud segmentation for diurnal and nocturnal ASI images than other four algorithms.

Download Full-text

THE LAND COVER CLASSIFICATION USING A FEATURE PYRAMID NETWORKS ARCHITECTURE FROM SATELLITE IMAGERY

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b3-2020-241-2020 ◽

2020 ◽

Vol XLIII-B3-2020 ◽

pp. 241-246

Author(s):

Q. Zhang ◽

Y. Zhang ◽

P. Yang ◽

Y. Meng ◽

S. Zhuo ◽

...

Keyword(s):

Land Cover ◽

Transfer Learning ◽

Satellite Imagery ◽

Loss Function ◽

Land Cover Classification ◽

Deep Convolutional Neural Networks ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Sensing Applications ◽

Feature Pyramid

Abstract. Extracting land cover information from satellite imagery is of great importance for the task of automated monitoring in various remote sensing applications. Deep convolutional neural networks make this task more feasible, but they are limited by the small dataset of annotated images. In this paper, we present a fully convolutional networks architecture, FPN-VGG, that combines Feature Pyramid Networks and VGG. In order to accomplish the task of land cover classification, we create a land cover dataset of pixel-wise annotated images, and employ a transfer learning step and the variant dice loss function to promote the performance of FPN-VGG. The results indicate that FPN-VGG shows more competence for land cover classification comparing with other state-of-the-art fully convolutional networks. The transfer learning and dice loss function are beneficial to improve the performance of on the small and unbalanced dataset. Our best model on the dataset gets an overall accuracy of 82.9%, an average F1 score of 66.0% and an average IoU of 52.7%.

Download Full-text