Unsupervised deep learning based change detection in Sentinel-2 images

Land cover and its change are crucial for many environmental applications. This study focuses on the land cover classification and change detection with multitemporal and multispectral Sentinel-2 satellite data. To address the challenging land cover change detection task, we rely on two different deep learning architectures and selected pre-processing steps. For example, we define an excluded class and deal with temporal water shoreline changes in the pre-processing. We employ a fully convolutional neural network (FCN), and we combine the FCN with long short-term memory (LSTM) networks. The FCN can only handle monotemporal input data, while the FCN combined with LSTM can use sequential information (multitemporal). Besides, we provided fixed and variable sequences as training sequences for the combined FCN and LSTM approach. The former refers to using six defined satellite images, while the latter consists of image sequences from an extended training pool of ten images. Further, we propose measures for the robustness concerning the selection of Sentinel-2 image data as evaluation metrics. We can distinguish between actual land cover changes and misclassifications of the deep learning approaches with these metrics. According to the provided metrics, both multitemporal LSTM approaches outperform the monotemporal FCN approach, about 3 to 5 percentage points (p.p.). The LSTM approach trained on the variable sequences detects 3 p.p. more land cover changes than the LSTM approach trained on the fixed sequences. Besides, applying our selected pre-processing improves the water classification and avoids reducing the dataset effectively by 17.6%. The presented LSTM approaches can be modified to provide applicability for a variable number of image sequences since we published the code of the deep learning models. The Sentinel-2 data and the ground truth are also freely available.

Download Full-text

Deep Learning for Regular Change Detection in Ukrainian Forest Ecosystem With Sentinel-2

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing ◽

10.1109/jstars.2020.3034186 ◽

2021 ◽

Vol 14 ◽

pp. 364-376

Author(s):

Kostiantyn Isaienkov ◽

Mykhailo Yushchuk ◽

Vladyslav Khramtsov ◽

Oleg Seliverstov

Keyword(s):

Deep Learning ◽

Change Detection ◽

Forest Ecosystem ◽

Regular Change ◽

Sentinel 2

Download Full-text

Burnt Forest Estimation from Sentinel-2 Imagery of Australia using Unsupervised Deep Learning

10.1109/dicta52665.2021.9647174 ◽

2021 ◽

Author(s):

Nosheen Abid ◽

Muhammad Imran Malik ◽

Muhammad Shahzad ◽

Faisal Shafait ◽

Haider Ali ◽

...

Keyword(s):

Deep Learning ◽

Unsupervised Deep Learning ◽

Sentinel 2

Download Full-text

Deep/Transfer Learning with Feature Space Ensemble Networks (FeatSpaceEnsNets) and Average Ensemble Networks (AvgEnsNets) for Change Detection Using DInSAR Sentinel-1 and Optical Sentinel-2 Satellite Data Fusion

Remote Sensing ◽

10.3390/rs13214394 ◽

2021 ◽

Vol 13 (21) ◽

pp. 4394

Author(s):

Zainoolabadien Karim ◽

Terence L. van Zyl

Keyword(s):

Deep Learning ◽

Change Detection ◽

Transfer Learning ◽

Binary Classification ◽

Feature Space ◽

Fine Tuning ◽

Urban Change ◽

Random Initialization ◽

Optical Imagery ◽

Sentinel 2

Differential interferometric synthetic aperture radar (DInSAR), coherence, phase, and displacement are derived from processing SAR images to monitor geological phenomena and urban change. Previously, Sentinel-1 SAR data combined with Sentinel-2 optical imagery has improved classification accuracy in various domains. However, the fusing of Sentinel-1 DInSAR processed imagery with Sentinel-2 optical imagery has not been thoroughly investigated. Thus, we explored this fusion in urban change detection by creating a verified balanced binary classification dataset comprising 1440 blobs. Machine learning models using feature descriptors and non-deep learning classifiers, including a two-layer convolutional neural network (ConvNet2), were used as baselines. Transfer learning by feature extraction (TLFE) using various pre-trained models, deep learning from random initialization, and transfer learning by fine-tuning (TLFT) were all evaluated. We introduce a feature space ensemble family (FeatSpaceEnsNet), an average ensemble family (AvgEnsNet), and a hybrid ensemble family (HybridEnsNet) of TLFE neural networks. The FeatSpaceEnsNets combine TLFE features directly in the feature space using logistic regression. AvgEnsNets combine TLFEs at the decision level by aggregation. HybridEnsNets are a combination of FeatSpaceEnsNets and AvgEnsNets. Several FeatSpaceEnsNets, AvgEnsNets, and HybridEnsNets, comprising a heterogeneous mixture of different depth and architecture models, are defined and evaluated. We show that, in general, TLFE outperforms both TLFT and classic deep learning for the small dataset used and that larger ensembles of TLFE models do not always improve accuracy. The best performing ensemble is an AvgEnsNet (84.862%) comprised of a ResNet50, ResNeXt50, and EfficientNet B4. This was matched by a similarly composed FeatSpaceEnsNet with an F1 score of 0.001 and variance of 0.266 less. The best performing HybridEnsNet had an accuracy of 84.775%. All of the ensembles evaluated outperform the best performing single model, ResNet50 with TLFE (83.751%), except for AvgEnsNet 3, AvgEnsNet 6, and FeatSpaceEnsNet 5. Five of the seven similarly composed FeatSpaceEnsNets outperform the corresponding AvgEnsNet.

Download Full-text

Unsupervised Deep Learning for Landslide Detection from Multispectral Sentinel-2 Imagery

Remote Sensing ◽

10.3390/rs13224698 ◽

2021 ◽

Vol 13 (22) ◽

pp. 4698

Author(s):

Hejar Shahabi ◽

Maryam Rahimzad ◽

Sepideh Tavakkoli Piralilou ◽

Omid Ghorbanzadeh ◽

Saied Homayouni ◽

...

Keyword(s):

Deep Learning ◽

Training Data ◽

New Approach ◽

Digital Elevation ◽

Unsupervised Deep Learning ◽

Elevation Model ◽

High Level ◽

The Impact ◽

Noise Fraction ◽

Sentinel 2

This paper proposes a new approach based on an unsupervised deep learning (DL) model for landslide detection. Recently, supervised DL models using convolutional neural networks (CNN) have been widely studied for landslide detection. Even though these models provide robust performance and reliable results, they depend highly on a large labeled dataset for their training step. As an alternative, in this paper, we developed an unsupervised learning model by employing a convolutional auto-encoder (CAE) to deal with the problem of limited labeled data for training. The CAE was used to learn and extract the abstract and high-level features without using training data. To assess the performance of the proposed approach, we used Sentinel-2 imagery and a digital elevation model (DEM) to map landslides in three different case studies in India, China, and Taiwan. Using minimum noise fraction (MNF) transformation, we reduced the multispectral dimension to three features containing more than 80% of scene information. Next, these features were stacked with slope data and NDVI as inputs to the CAE model. The Huber reconstruction loss was used to evaluate the inputs. We achieved reconstruction losses ranging from 0.10 to 0.147 for the MNF features, slope, and NDVI stack for all three study areas. The mini-batch K-means clustering method was used to cluster the features into two to five classes. To evaluate the impact of deep features on landslide detection, we first clustered a stack of MNF features, slope, and NDVI, then the same ones plus with the deep features. For all cases, clustering based on deep features provided the highest precision, recall, F1-score, and mean intersection over the union in landslide detection.

Download Full-text

Uni-Temporal Multispectral Imagery for Burned Area Mapping with Deep Learning

Remote Sensing ◽

10.3390/rs13081509 ◽

2021 ◽

Vol 13 (8) ◽

pp. 1509

Author(s):

Xikun Hu ◽

Yifang Ban ◽

Andrea Nascetti

Keyword(s):

Deep Learning ◽

Large Scale ◽

Multiple Scales ◽

Detection Methods ◽

Burned Area ◽

Landsat 8 ◽

Multispectral Imagery ◽

Burned Areas ◽

Local Climate Zones ◽

Sentinel 2

Accurate burned area information is needed to assess the impacts of wildfires on people, communities, and natural ecosystems. Various burned area detection methods have been developed using satellite remote sensing measurements with wide coverage and frequent revisits. Our study aims to expound on the capability of deep learning (DL) models for automatically mapping burned areas from uni-temporal multispectral imagery. Specifically, several semantic segmentation network architectures, i.e., U-Net, HRNet, Fast-SCNN, and DeepLabv3+, and machine learning (ML) algorithms were applied to Sentinel-2 imagery and Landsat-8 imagery in three wildfire sites in two different local climate zones. The validation results show that the DL algorithms outperform the ML methods in two of the three cases with the compact burned scars, while ML methods seem to be more suitable for mapping dispersed burn in boreal forests. Using Sentinel-2 images, U-Net and HRNet exhibit comparatively identical performance with higher kappa (around 0.9) in one heterogeneous Mediterranean fire site in Greece; Fast-SCNN performs better than others with kappa over 0.79 in one compact boreal forest fire with various burn severity in Sweden. Furthermore, directly transferring the trained models to corresponding Landsat-8 data, HRNet dominates in the three test sites among DL models and can preserve the high accuracy. The results demonstrated that DL models can make full use of contextual information and capture spatial details in multiple scales from fire-sensitive spectral bands to map burned areas. Using only a post-fire image, the DL methods not only provide automatic, accurate, and bias-free large-scale mapping option with cross-sensor applicability, but also have potential to be used for onboard processing in the next Earth observation satellites.

Download Full-text

Unsupervised Deep Learning For Accelerated High Quality Echocardiography

2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) ◽

10.1109/isbi48211.2021.9433770 ◽

2021 ◽

Author(s):

Shujaat Khan ◽

Jaeyoung Huh ◽

Jong Chul Ye

Keyword(s):

Deep Learning ◽

High Quality ◽

Unsupervised Deep Learning

Download Full-text

Evaluating the Potentiality of Sentinel-2 for Change Detection Analysis Associated to LULUCF in Wallonia, Belgium

Land ◽

10.3390/land10010055 ◽

2021 ◽

Vol 10 (1) ◽

pp. 55

Author(s):

Odile Close ◽

Sophie Petit ◽

Benjamin Beaumont ◽

Eric Hallot

Keyword(s):

Climate Change ◽

Land Use ◽

Change Detection ◽

Signal To Noise Ratio ◽

Land Conversion ◽

Sustainable Resources ◽

Detection Techniques ◽

Detection Analysis ◽

Conversion Rates ◽

Sentinel 2

Land Use/Cover changes are crucial for the use of sustainable resources and the delivery of ecosystem services. They play an important contribution in the climate change mitigation due to their ability to emit and remove greenhouse gas from the atmosphere. These emissions/removals are subject to an inventory which must be reported annually under the United Nations Framework Convention on Climate Change. This study investigates the use of Sentinel-2 data for analysing lands conversion associated to Land Use, Land Use Change and Forestry sector in the Wallonia region (southern Belgium). This region is characterized by one of the lowest conversion rates across European countries, which constitutes a particular challenge in identifying land changes. The proposed research tests the most commonly used change detection techniques on a bi-temporal and multi-temporal set of mosaics of Sentinel-2 data from the years 2016 and 2018. Our results reveal that land conversion is a very rare phenomenon in Wallonia. All the change detection techniques tested have been found to substantially overestimate the changes. In spite of this moderate results our study has demonstrated the potential of Sentinel-2 regarding land conversion. However, in this specific context of very low magnitude of land conversion in Wallonia, change detection techniques appear to be not sufficient to exceed the signal to noise ratio.

Download Full-text

Representation Learning for Fine-Grained Change Detection

Sensors ◽

10.3390/s21134486 ◽

2021 ◽

Vol 21 (13) ◽

pp. 4486

Author(s):

Niall O’Mahony ◽

Sean Campbell ◽

Lenka Krpalkova ◽

Anderson Carvalho ◽

Joseph Walsh ◽

...

Keyword(s):

Deep Learning ◽

Change Detection ◽

Model Calibration ◽

State Of The Art ◽

Representation Learning ◽

Machine Intelligence ◽

The State ◽

Sensor Data ◽

Fine Grained ◽

Learning Techniques

Fine-grained change detection in sensor data is very challenging for artificial intelligence though it is critically important in practice. It is the process of identifying differences in the state of an object or phenomenon where the differences are class-specific and are difficult to generalise. As a result, many recent technologies that leverage big data and deep learning struggle with this task. This review focuses on the state-of-the-art methods, applications, and challenges of representation learning for fine-grained change detection. Our research focuses on methods of harnessing the latent metric space of representation learning techniques as an interim output for hybrid human-machine intelligence. We review methods for transforming and projecting embedding space such that significant changes can be communicated more effectively and a more comprehensive interpretation of underlying relationships in sensor data is facilitated. We conduct this research in our work towards developing a method for aligning the axes of latent embedding space with meaningful real-world metrics so that the reasoning behind the detection of change in relation to past observations may be revealed and adjusted. This is an important topic in many fields concerned with producing more meaningful and explainable outputs from deep learning and also for providing means for knowledge injection and model calibration in order to maintain user confidence.

Download Full-text