Refined UNet V4: End-to-End Patch-Wise Network for Cloud and Shadow Segmentation with Bilateral Grid

Remote sensing images are usually contaminated by cloud and corresponding shadow regions, making cloud and shadow detection one of the essential prerequisites for processing and translation of remote sensing images. Edge-precise cloud and shadow segmentation remains challenging due to the inherent high-level semantic acquisition of current neural segmentation fashions. We, therefore, introduce the Refined UNet series to partially achieve edge-precise cloud and shadow detection, including two-stage Refined UNet, v2 with a potentially efficient gray-scale guided Gaussian filter-based CRF, and v3 with an efficient multi-channel guided Gaussian filter-based CRF. However, it is visually demonstrated that the locally linear kernel used in v2 and v3 is not sufficiently sensitive to potential edges in comparison with Refined UNet. Accordingly, we turn back to the investigation of an end-to-end UNet-CRF architecture with a Gaussian-form bilateral kernel and its relatively efficient approximation. In this paper, we present Refined UNet v4, an end-to-end edge-precise segmentation network for cloud and shadow detection, which is capable of retrieving regions of interest with relatively tight edges and potential shadow regions with ambiguous edges. Specifically, we inherit the UNet-CRF architecture exploited in the Refined UNet series, which concatenates a UNet backbone of coarsely locating cloud and shadow regions and an embedded CRF layer of refining edges. In particular, the bilateral grid-based approximation to the Gaussian-form bilateral kernel is applied to the bilateral message-passing step, in order to ensure the delineation of sufficiently tight edges and the retrieval of shadow regions with ambiguous edges. Our TensorFlow implementation of the bilateral approximation is relatively computationally efficient in comparison with Refined UNet, attributed to the straightforward GPU acceleration. Extensive experiments on Landsat 8 OLI dataset illustrate that our v4 can achieve edge-precise cloud and shadow segmentation and improve the retrieval of shadow regions, and also confirm its computational efficiency.

BSIRNet: A Road Extraction Network with Bidirectional Spatial Information Reasoning

Automatic extraction of road information from remote sensing images is widely used in many fields, such as urban planning and automatic navigation. However, due to interference from noise and occlusion, the existing road extraction methods can easily lead to road discontinuity. To solve this problem, a road extraction network with bidirectional spatial information reasoning (BSIRNet) is proposed, in which neighbourhood feature fusion is used to capture spatial context dependencies and expand the receptive field, and an information processing unit with a recurrent neural network structure is used to capture channel dependencies. BSIRNet enhances the connectivity of road information through spatial information reasoning. Using the public Massachusetts road dataset and Wuhan University road dataset, the superiority of the proposed method is verified by comparing its results with those of other models.

Comparison of Classical Methods and Mask R-CNN for Automatic Tree Detection and Mapping Using UAV Imagery

Detecting and mapping individual trees accurately and automatically from remote sensing images is of great significance for precision forest management. Many algorithms, including classical methods and deep learning techniques, have been developed and applied for tree crown detection from remote sensing images. However, few studies have evaluated the accuracy of different individual tree detection (ITD) algorithms and their data and processing requirements. This study explored the accuracy of ITD using local maxima (LM) algorithm, marker-controlled watershed segmentation (MCWS), and Mask Region-based Convolutional Neural Networks (Mask R-CNN) in a young plantation forest with different test images. Manually delineated tree crowns from UAV imagery were used for accuracy assessment of the three methods, followed by an evaluation of the data processing and application requirements for three methods to detect individual trees. Overall, Mask R-CNN can best use the information in multi-band input images for detecting individual trees. The results showed that the Mask R-CNN model with the multi-band combination produced higher accuracy than the model with a single-band image, and the RGB band combination achieved the highest accuracy for ITD (F1 score = 94.68%). Moreover, the Mask R-CNN models with multi-band images are capable of providing higher accuracies for ITD than the LM and MCWS algorithms. The LM algorithm and MCWS algorithm also achieved promising accuracies for ITD when the canopy height model (CHM) was used as the test image (F1 score = 87.86% for LM algorithm, F1 score = 85.92% for MCWS algorithm). The LM and MCWS algorithms are easy to use and lower computer computational requirements, but they are unable to identify tree species and are limited by algorithm parameters, which need to be adjusted for each classification. It is highlighted that the application of deep learning with its end-to-end-learning approach is very efficient and capable of deriving the information from multi-layer images, but an additional training set is needed for model training, robust computer resources are required, and a large number of accurate training samples are necessary. This study provides valuable information for forestry practitioners to select an optimal approach for detecting individual trees.

Variational-Scale Segmentation for Multispectral Remote-Sensing Images Using Spectral Indices

Many studies have focused on performing variational-scale segmentation to represent various geographical objects in high-resolution remote-sensing images. However, it remains a significant challenge to select the most appropriate scales based on the geographical-distribution characteristics of ground objects. In this study, we propose a variational-scale multispectral remote-sensing image segmentation method using spectral indices. Real scenes in remote-sensing images contain different types of land cover with different scales. Therefore, it is difficult to segment images optimally based on the scales of different ground objects. To guarantee image segmentation of ground objects with their own scale information, spectral indices that can be used to enhance some types of land cover, such as green cover and water bodies, were introduced into marker generation for the watershed transformation. First, a vector field model was used to determine the gradient of a multispectral remote-sensing image, and a marker was generated from the gradient. Second, appropriate spectral indices were selected, and the kernel density estimation was used to generate spectral-index marker images based on the analysis of spectral indices. Third, a series of mathematical morphology operations were used to obtain a combined marker image from the gradient and the spectral index markers. Finally, the watershed transformation was used for image segmentation. In a segmentation experiment, an optimal threshold for the spectral-index-marker generation method was identified. Additionally, the influence of the scale parameter was analyzed in a segmentation experiment based on a five-subset dataset. The comparative results for the proposed method, the commonly used watershed segmentation method, and the multiresolution segmentation method demonstrate that the proposed method yielded multispectral remote-sensing images with much better performance than the other methods.

A Neural Network-Based Distributed Compressed Sensing Algorithm to Derive Continuous Hyperspectral Images in Wireless Sensor Networks

Hyperspectral imaging is a versatile and powerful technology for gathering geo-data. Planes and satellites equipped with hyperspectral cameras are currently the leading contenders for large-scale imaging projects. Aiming at the shortcomings of traditional methods for detecting sparse representation of multi-spectral images, this paper proposes wireless sensor networks (WSNs) based single-hyperspectral image super-resolution method based on deep residual convolutional neural networks. We propose a different strategy that involves merging cheaper multispectral sensors to achieve hyperspectral-like spectral resolution while maintaining the WSN's spatial resolution. This method studies and mines the nonlinear relationship between low-resolution remote sensing images and high-resolution remote sensing images, constructs a deep residual convolutional neural network, connects multiple residual blocks in series, and removes some unnecessary modules. For this purpose, a decision support system is used that provides the outcome to the next layer. Finally, this paper, fully explores the similarities between natural images and hyperspectral images, use natural image samples to train convolutional neural networks, and further use migration learning to introduce the trained network model to the super-resolution problem of high-resolution remote sensing images, and solve the lack of training samples problem. A comparison between different algorithms for processing data on datasets collected in situ and via remote sensing is used to evaluate the proposed approach. The experimental results show that the method has good performance and can obtain better super-resolution effects.

Nature-Based Solutions Benefit the Economic–Ecological Coordination of Pastoral Areas: An Outstanding Herdsman’s Experience in Xilin Gol, China

Grassland has always had a difficult economic–ecological relationship, as coordination between its ecological conservation and the sustainable development of animal husbandry is required. Nature-based Solutions (NbS), who make full use of the natural ecosystem services, have successfully solved some economic–ecological issues, but still have unclear implementation prospects for grassland management. The Xilin Gol grassland is one of the most typical pastoral areas in China; there is a village chief named Bateer, who has already used NbS for grassland management. To confirm whether the solutions employed by Bateer have been effective for both increasing economic profits and protecting grassland ecosystem, we interviewed him, and many other herdsmen, using questionnaires about their livelihood. Based on these questionnaires, we calculated and compared their income–cost ratios. Meanwhile, we analyzed the NDVI variations inside their rangelands through high-resolution remote sensing images. The results showed that the herdsmen in Bateer’s village had a much higher disposable income and income–cost ratio than others, and their rangelands also had a higher value and a more obvious increasing trend of NDVI. Bateer’s success proves that the NbS can also play a positive role in grassland management, which can provide a valuable guidance for economic–ecological coordination in pastoral areas.