scholarly journals Transferability of the Deep Learning Mask R-CNN Model for Automated Mapping of Ice-Wedge Polygons in High-Resolution Satellite and UAV Images

2020 ◽  
Vol 12 (7) ◽  
pp. 1085 ◽  
Author(s):  
Weixing Zhang ◽  
Anna K. Liljedahl ◽  
Mikhail Kanevskiy ◽  
Howard E. Epstein ◽  
Benjamin M. Jones ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Spatial Resolution ◽  
Image Resolution ◽  
Aerial Imagery ◽  
The Arctic ◽  
Sensor Platforms ◽  
Cmos Sensor ◽  
Ice Wedge ◽  
Uav Images

State-of-the-art deep learning technology has been successfully applied to relatively small selected areas of very high spatial resolution (0.15 and 0.25 m) optical aerial imagery acquired by a fixed-wing aircraft to automatically characterize ice-wedge polygons (IWPs) in the Arctic tundra. However, any mapping of IWPs at regional to continental scales requires images acquired on different sensor platforms (particularly satellite) and a refined understanding of the performance stability of the method across sensor platforms through reliable evaluation assessments. In this study, we examined the transferability of a deep learning Mask Region-Based Convolutional Neural Network (R-CNN) model for mapping IWPs in satellite remote sensing imagery (~0.5 m) covering 272 km2 and unmanned aerial vehicle (UAV) (0.02 m) imagery covering 0.32 km2. Multi-spectral images were obtained from the WorldView-2 satellite sensor and pan-sharpened to ~0.5 m, and a 20 mp CMOS sensor camera onboard a UAV, respectively. The training dataset included 25,489 and 6022 manually delineated IWPs from satellite and fixed-wing aircraft aerial imagery near the Arctic Coastal Plain, northern Alaska. Quantitative assessments showed that individual IWPs were correctly detected at up to 72% and 70%, and delineated at up to 73% and 68% F1 score accuracy levels for satellite and UAV images, respectively. Expert-based qualitative assessments showed that IWPs were correctly detected at good (40–60%) and excellent (80–100%) accuracy levels for satellite and UAV images, respectively, and delineated at excellent (80–100%) level for both images. We found that (1) regardless of spatial resolution and spectral bands, the deep learning Mask R-CNN model effectively mapped IWPs in both remote sensing satellite and UAV images; (2) the model achieved a better accuracy in detection with finer image resolution, such as UAV imagery, yet a better accuracy in delineation with coarser image resolution, such as satellite imagery; (3) increasing the number of training data with different resolutions between the training and actual application imagery does not necessarily result in better performance of the Mask R-CNN in IWPs mapping; (4) and overall, the model underestimates the total number of IWPs particularly in terms of disjoint/incomplete IWPs.

scholarly journals Deep Learning Approaches for the Mapping of Tree Species Diversity in a Tropical Wetland Using Airborne LiDAR and High-Spatial-Resolution Remote Sensing Images

Forests ◽  
2019 ◽  
Vol 10 (11) ◽  
pp. 1047 ◽  
Author(s):  
Ying Sun ◽  
Jianfeng Huang ◽  
Zurui Ao ◽  
Dazhao Lao ◽  
Qinchuan Xin
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Species Diversity ◽  
Spatial Resolution ◽  
Tree Species ◽  
High Spatial Resolution ◽  
Diversity Index ◽  
Tree Species Diversity ◽  
Individual Tree ◽  
Rgb Images

The monitoring of tree species diversity is important for forest or wetland ecosystem service maintenance or resource management. Remote sensing is an efficient alternative to traditional field work to map tree species diversity over large areas. Previous studies have used light detection and ranging (LiDAR) and imaging spectroscopy (hyperspectral or multispectral remote sensing) for species richness prediction. The recent development of very high spatial resolution (VHR) RGB images has enabled detailed characterization of canopies and forest structures. In this study, we developed a three-step workflow for mapping tree species diversity, the aim of which was to increase knowledge of tree species diversity assessment using deep learning in a tropical wetland (Haizhu Wetland) in South China based on VHR-RGB images and LiDAR points. Firstly, individual trees were detected based on a canopy height model (CHM, derived from LiDAR points) by the local-maxima-based method in the FUSION software (Version 3.70, Seattle, USA). Then, tree species at the individual tree level were identified via a patch-based image input method, which cropped the RGB images into small patches (the individually detected trees) based on the tree apexes detected. Three different deep learning methods (i.e., AlexNet, VGG16, and ResNet50) were modified to classify the tree species, as they can make good use of the spatial context information. Finally, four diversity indices, namely, the Margalef richness index, the Shannon–Wiener diversity index, the Simpson diversity index, and the Pielou evenness index, were calculated from the fixed subset with a size of 30 × 30 m for assessment. In the classification phase, VGG16 had the best performance, with an overall accuracy of 73.25% for 18 tree species. Based on the classification results, mapping of tree species diversity showed reasonable agreement with field survey data (R2Margalef = 0.4562, root-mean-square error RMSEMargalef = 0.5629; R2Shannon–Wiener = 0.7948, RMSEShannon–Wiener = 0.7202; R2Simpson = 0.7907, RMSESimpson = 0.1038; and R2Pielou = 0.5875, RMSEPielou = 0.3053). While challenges remain for individual tree detection and species classification, the deep-learning-based solution shows potential for mapping tree species diversity.

scholarly journals Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement

2020 ◽  
Vol 12 (5) ◽  
pp. 758 ◽  
Author(s):  
Mengjiao Qin ◽  
Sébastien Mavromatis ◽  
Linshu Hu ◽  
Feng Zhang ◽  
Renyi Liu ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Spatial Resolution ◽  
Super Resolution ◽  
Image Resolution ◽  
Single Image ◽  
Remote Sensing Images ◽  
Practical Applications ◽  
Specific Enhancement ◽  
Urban Monitoring ◽  
High Resolution Images

Super-resolution (SR) is able to improve the spatial resolution of remote sensing images, which is critical for many practical applications such as fine urban monitoring. In this paper, a new single-image SR method, deep gradient-aware network with image-specific enhancement (DGANet-ISE) was proposed to improve the spatial resolution of remote sensing images. First, DGANet was proposed to model the complex relationship between low- and high-resolution images. A new gradient-aware loss was designed in the training phase to preserve more gradient details in super-resolved remote sensing images. Then, the ISE approach was proposed in the testing phase to further improve the SR performance. By using the specific features of each test image, ISE can further boost the generalization capability and adaptability of our method on inexperienced datasets. Finally, three datasets were used to verify the effectiveness of our method. The results indicate that DGANet-ISE outperforms the other 14 methods in the remote sensing image SR, and the cross-database test results demonstrate that our method exhibits satisfactory generalization performance in adapting to new data.

scholarly journals Application of Deep Learning for Delineation of Visible Cadastral Boundaries from Remote Sensing Imagery

2019 ◽  
Vol 11 (21) ◽  
pp. 2505 ◽  
Author(s):  
Crommelinck ◽  
Koeva ◽  
Yang ◽  
Vosselman
Keyword(s):  
Remote Sensing ◽  
Image Segmentation ◽  
Image Resolution ◽  
Aerial Imagery ◽  
Boundary Line ◽  
Data Sets ◽  
Remote Sensing Imagery ◽  
Manual Delineation ◽  
Segmentation Boundary ◽  
Boundary Classification

Cadastral boundaries are often demarcated by objects that are visible in remote sensing imagery. Indirect surveying relies on the delineation of visible parcel boundaries from such images. Despite advances in automated detection and localization of objects from images, indirect surveying is rarely automated and relies on manual on-screen delineation. We have previously introduced a boundary delineation workflow, comprising image segmentation, boundary classification and interactive delineation that we applied on Unmanned Aerial Vehicle (UAV) data to delineate roads. In this study, we improve each of these steps. For image segmentation, we remove the need to reduce the image resolution and we limit over-segmentation by reducing the number of segment lines by 80% through filtering. For boundary classification, we show how Convolutional Neural Networks (CNN) can be used for boundary line classification, thereby eliminating the previous need for Random Forest (RF) feature generation and thus achieving 71% accuracy. For interactive delineation, we develop additional and more intuitive delineation functionalities that cover more application cases. We test our approach on more varied and larger data sets by applying it to UAV and aerial imagery of 0.02–0.25 m resolution from Kenya, Rwanda and Ethiopia. We show that it is more effective in terms of clicks and time compared to manual delineation for parcels surrounded by visible boundaries. Strongest advantages are obtained for rural scenes delineated from aerial imagery, where the delineation effort per parcel requires 38% less time and 80% fewer clicks compared to manual delineation.

scholarly journals Intelligent Object Recognition of Urban Water Bodies Based on Deep Learning for Multi-Source and Multi-Temporal High Spatial Resolution Remote Sensing Imagery

Sensors ◽  
2020 ◽  
Vol 20 (2) ◽  
pp. 397 ◽  
Author(s):  
Shiran Song ◽  
Jianhua Liu ◽  
Yuan Liu ◽  
Guoqiang Feng ◽  
Hui Han ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Object Detection ◽  
Spatial Resolution ◽  
Water Body ◽  
High Spatial Resolution ◽  
Water Bodies ◽  
Top Down ◽  
Multi Temporal ◽  
Body Recognition

High spatial resolution remote sensing image (HSRRSI) data provide rich texture, geometric structure, and spatial distribution information for surface water bodies. The rich detail information provides better representation of the internal components of each object category and better reflects the relationships between adjacent objects. In this context, recognition methods such as geographic object-based image analysis (GEOBIA) have improved significantly. However, these methods focus mainly on bottom-up classifications from visual features to semantic categories, but ignore top-down feedback which can optimize recognition results. In recent years, deep learning has been applied in the field of remote sensing measurements because of its powerful feature extraction ability. A special convolutional neural network (CNN) based region proposal generation and object detection integrated framework has greatly improved the performance of object detection for HSRRSI, which provides a new method for water body recognition based on remote sensing data. This study uses the excellent “self-learning ability” of deep learning to construct a modified structure of the Mask R-CNN method which integrates bottom-up and top-down processes for water recognition. Compared with traditional methods, our method is completely data-driven without prior knowledge, and it can be regarded as a novel technical procedure for water body recognition in practical engineering application. Experimental results indicate that the method produces accurate recognition results for multi-source and multi-temporal water bodies, and can effectively avoid confusion with shadows and other ground features.

scholarly journals Self-Organizing Deep Learning (SO-UNet)—A Novel Framework to Classify Urban and Peri-Urban Forests

2021 ◽  
Vol 13 (10) ◽  
pp. 5548
Author(s):  
Mohamad M. Awad ◽  
Marco Lauteri
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Spatial Resolution ◽  
Forest Type ◽  
Urban Forests ◽  
Learning Technologies ◽  
Aerial Images ◽  
Hyper Spectral ◽  
Type Classification ◽  
Self Organizing

Forest-type classification is a very complex and difficult subject. The complexity increases with urban and peri-urban forests because of the variety of features that exist in remote sensing images. The success of forest management that includes forest preservation depends strongly on the accuracy of forest-type classification. Several classification methods are used to map urban and peri-urban forests and to identify healthy and non-healthy ones. Some of these methods have shown success in the classification of forests where others failed. The successful methods used specific remote sensing data technology, such as hyper-spectral and very high spatial resolution (VHR) images. However, both VHR and hyper-spectral sensors are very expensive, and hyper-spectral sensors are not widely available on satellite platforms, unlike multi-spectral sensors. Moreover, aerial images are limited in use, very expensive, and hard to arrange and manage. To solve the aforementioned problems, an advanced method, self-organizing–deep learning (SO-UNet), was created to classify forests in the urban and peri-urban environment using multi-spectral, multi-temporal, and medium spatial resolution Sentinel-2 images. SO-UNet is a combination of two different machine learning technologies: artificial neural network unsupervised self-organizing maps and deep learning UNet. Many experiments have been conducted, and the results showed that SO-UNet overwhelms UNet significantly. The experiments encompassed different settings for the parameters that control the algorithms.

scholarly journals Arctic sea ice cover data from spaceborne SAR by deep learning

2020 ◽  
Author(s):  
Yi-Ran Wang ◽  
Xiao-Ming Li
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Sea Ice ◽  
Spatial Resolution ◽  
Ice Cover ◽  
Arctic Sea Ice ◽  
The Arctic ◽  
Absolute Difference ◽  
Sea Ice Concentration ◽  
Ice Concentration

Abstract. Widely used sea ice concentration and sea ice cover in polar regions are derived mainly from spaceborne microwave radiometer and scatterometer data, and the typical spatial resolution of these products ranges from several to dozens of kilometers. Due to dramatic changes in polar sea ice, high-resolution sea ice cover data are drawing increasing attention for polar navigation, environmental research, and offshore operations. In this paper, we focused on developing an approach for deriving a high-resolution sea ice cover product for the Arctic using Sentinel-1 (S1) dual-polarization (horizontal-horizontal, HH, and horizontal-vertical, HV) data in extra wide swath (EW) mode. The approach for discriminating sea ice from open water by synthetic aperture radar (SAR) data is based on a modified U-Net architecture, a deep learning network. By employing an integrated stacking model to combine multiple U-Net classifiers with diverse specializations, sea ice segmentation is achieved with superior accuracy over any individual classifier. We applied the proposed approach to over 28,000 S1 EW images acquired in 2019 to obtain sea ice cover products in a high spatial resolution of 400 m. By converting the S1-derived sea ice cover to concentration and then compared with Advanced Microwave Scanning Radiometer 2 (AMSR2) sea ice concentration data, showing an average absolute difference of 5.55 % with seasonal fluctuations. A direct comparison with Interactive Multisensor Snow and Ice Mapping System (IMS) daily sea ice cover data achieves an average accuracy of 93.98 %. These results show that the developed S1-derived sea ice cover results are comparable to the AMSR and IMS data in terms of overall accuracy but superior to these data in presenting detailed sea ice cover information, particularly in the marginal ice zone (MIZ). Data are available at: https://doi.org/10.11922/sciencedb.00273 (Wang and Li, 2020).

Deep Learning Enhances the Detection of Breeding Birds in UAV Images

2021 ◽  
Author(s):  
Benjamin Kellenberger ◽  
Thor Veen ◽  
Eelke Folmer ◽  
Devis Tuia
Keyword(s):  
Machine Learning ◽  
Remote Sensing ◽  
Deep Learning ◽  
High Resolution ◽  
Large Scale ◽  
Bird Species ◽  
Decision Fusion ◽  
Training Data ◽  
Seabird Species ◽  
Uav Images

<p>Recently, Unmanned Aerial Vehicles (UAVs) equipped with high-resolution imaging sensors have become a viable alternative for ecologists to conduct wildlife censuses, compared to foot surveys. They cause less disturbance by sensing remotely, they provide coverage of otherwise inaccessible areas, and their images can be reviewed and double-checked in controlled screening sessions. However, the amount of data they generate often makes this photo-interpretation stage prohibitively time-consuming.</p><p>In this work, we automate the detection process with deep learning [4]. We focus on counting coastal seabirds on sand islands off the West African coast, where species like the African Royal Tern are at the top of the food chain [5]. Monitoring their abundance provides invaluable insights into biodiversity in this area [7]. In a first step, we obtained orthomosaics from nadir-looking UAVs over six sand islands with 1cm resolution. We then fully labelled one of them with points for four seabird species, which required three weeks for five annotators to do and resulted in over 21,000 individuals. Next, we further labelled the other five orthomosaics, but in an incomplete manner; we aimed for a low number of only 200 points per species. These points, together with a few background polygons, served as training data for our ResNet-based [2] detection model. This low number of points required multiple strategies to obtain stable predictions, including curriculum learning [1] and post-processing by a Markov random field [6]. In the end, our model was able to accurately predict the 21,000 birds of the test image with 90% precision at 90% recall (Fig. 1) [3]. Furthermore, this model required a mere 4.5 hours from creating training data to the final prediction, which is a fraction of the three weeks needed for the manual labelling process. Inference time is only a few minutes, which makes the model scale favourably to many more islands. In sum, the combination of UAVs and machine learning-based detectors simultaneously provides census possibilities with unprecedentedly high accuracy and comparably minuscule execution time.</p><p><img src="https://contentmanager.copernicus.org/fileStorageProxy.php?f=gnp.bc5211f4f60067568601161/sdaolpUECMynit/12UGE&app=m&a=0&c=eeda7238e992b9591c2fec19197f67dc&ct=x&pn=gnp.elif&d=1" alt=""></p><p><em>Fig. 1: Our model is able to predict over 21,000 birds in high-resolution UAV images in a fraction of time compared to weeks of manual labelling.</em></p><p> </p><p>References</p><p>1. Bengio, Yoshua, et al. "Curriculum learning." Proceedings of the 26th annual international conference on machine learning. 2009.</p><p>2. He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.</p><p>3. Kellenberger, Benjamin, et al. “21,000 Birds in 4.5 Hours: Efficient Large-scale Seabird Detection with Machine Learning.” Remote Sensing in Ecology and Conservation. Under review.</p><p>4. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436-444.</p><p>5. Parsons, Matt, et al. "Seabirds as indicators of the marine environment." ICES Journal of Marine Science 65.8 (2008): 1520-1526.</p><p>6. Tuia, Devis, Michele Volpi, and Gabriele Moser. "Decision fusion with multiple spatial supports by conditional random fields." IEEE Transactions on Geoscience and Remote Sensing 56.6 (2018): 3277-3289.</p><p>7. Veen, Jan, Hanneke Dallmeijer, and Thor Veen. "Selecting piscivorous bird species for monitoring environmental change in the Banc d'Arguin, Mauritania." Ardea 106.1 (2018): 5-18.</p>

Sign in / Sign up

Export Citation Format

Share Document