Super-Resolution of Sentinel-2 Images Using Convolutional Neural Networks and Real Ground Truth Data

Earth observation data is becoming more accessible and affordable thanks to the Copernicus programme and its Sentinel missions. Every location worldwide can be freely monitored approximately every 5 days using the multi-spectral images provided by Sentinel-2. The spatial resolution of these images for RGBN (RGB + Near-infrared) bands is 10 m, which is more than enough for many tasks but falls short for many others. For this reason, if their spatial resolution could be enhanced without additional costs, any posterior analyses based on these images would be benefited. Previous works have mainly focused on increasing the resolution of lower resolution bands of Sentinel-2 (20 m and 60 m) to 10 m resolution. In these cases, super-resolution is supported by bands captured at finer resolutions (RGBN at 10 m). On the contrary, this paper focuses on the problem of increasing the spatial resolution of 10 m bands to either 5 m or 2.5 m resolutions, without having additional information available. This problem is known as single-image super-resolution. For standard images, deep learning techniques have become the de facto standard to learn the mapping from lower to higher resolution images due to their learning capacity. However, super-resolution models learned for standard images do not work well with satellite images and hence, a specific model for this problem needs to be learned. The main challenge that this paper aims to solve is how to train a super-resolution model for Sentinel-2 images when no ground truth exists (Sentinel-2 images at 5 m or 2.5 m). Our proposal consists of using a reference satellite with a high similarity in terms of spectral bands with respect to Sentinel-2, but with higher spatial resolution, to create image pairs at both the source and target resolutions. This way, we can train a state-of-the-art Convolutional Neural Network to recover details not present in the original RGBN bands. An exhaustive experimental study is carried out to validate our proposal, including a comparison with the most extended strategy for super-resolving Sentinel-2, which consists in learning a model to super-resolve from an under-sampled version at either 40 m or 20 m to the original 10 m resolution and then, applying this model to super-resolve from 10 m to 5 m or 2.5 m. Finally, we will also show that the spectral radiometry of the native bands is maintained when super-resolving images, in such a way that they can be used for any subsequent processing as if they were images acquired by Sentinel-2.

Download Full-text

LEARNING SUPER-RESOLUTION FOR SENTINEL-2 IMAGES WITH REAL GROUND TRUTH DATA FROM A REFERENCE SATELLITE

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-1-2020-9-2020 ◽

2020 ◽

Vol V-1-2020 ◽

pp. 9-16

Author(s):

M. Galar ◽

R. Sesma ◽

C. Ayala ◽

L. Albizua ◽

C. Aranda

Keyword(s):

Deep Learning ◽

Spatial Resolution ◽

Near Infrared ◽

State Of The Art ◽

Super Resolution ◽

Ground Truth ◽

Ground Truth Data ◽

Additional Information ◽

Image Super Resolution ◽

Sentinel 2

Abstract. Copernicus program via its Sentinel missions is making earth observation more accessible and affordable for everybody. Sentinel-2 images provide multi-spectral information every 5 days for each location. However, the maximum spatial resolution of its bands is 10m for RGB and near-infrared bands. Increasing the spatial resolution of Sentinel-2 images without additional costs, would make any posterior analysis more accurate. Most approaches on super-resolution for Sentinel-2 have focused on obtaining 10m resolution images for those at lower resolutions (20m and 60m), taking advantage of the information provided by bands of finer resolutions (10m). Otherwise, our focus is on increasing the resolution of the 10m bands, that is, super-resolving 10m bands to 2.5m resolution, where no additional information is available. This problem is known as single-image super-resolution and deep learning-based approaches have become the state-of-the-art for this problem on standard images. Obviously, models learned for standard images do not translate well to satellite images. Hence, the problem is how to train a deep learning model for super-resolving Sentinel-2 images when no ground truth exist (Sentinel-2 images at 2.5m). We propose a methodology for learning Convolutional Neural Networks for Sentinel-2 image super-resolution making use of images from other sensors having a high similarity with Sentinel-2 in terms of spectral bands, but greater spatial resolution. Our proposal is tested with a state-of-the-art neural network showing that it can be useful for learning to increase the spatial resolution of RGB and near-infrared bands of Sentinel-2.

Download Full-text

Applying the Tropical Peatland Combustion Algorithm to Landsat-8 Operational Land Imager (OLI) and Sentinel-2 Multi Spectral Instrument (MSI) Imagery

Remote Sensing ◽

10.3390/rs12233958 ◽

2020 ◽

Vol 12 (23) ◽

pp. 3958

Author(s):

Parwati Sofan ◽

David Bruce ◽

Eriita Jones ◽

M. Rokhis Khomarudin ◽

Orbita Roswintiarti

Keyword(s):

Spatial Resolution ◽

Fire Detection ◽

Ground Truth ◽

Thermal Infrared ◽

Landsat 8 ◽

Ground Truth Data ◽

Spectral Instrument ◽

Tropical Peatland ◽

Tropical Environments ◽

Sentinel 2

This study establishes a new technique for peatland fire detection in tropical environments using Landsat-8 and Sentinel-2. The Tropical Peatland Combustion Algorithm (ToPeCAl) without longwave thermal infrared (TIR) (henceforth known as ToPeCAl-2) was tested on Landsat-8 Operational Land Imager (OLI) data and then applied to Sentinel-2 Multi Spectral Instrument (MSI) data. The research is aimed at establishing peatland fire information at higher spatial resolution and more frequent observation than from Landsat-8 data over Indonesia’s peatlands. ToPeCAl-2 applied to Sentinel-2 was assessed by comparing fires detected from the original ToPeCAl applied to Landsat-8 OLI/Thermal Infrared Sensor (TIRS) verified through comparison with ground truth data. An adjustment of ToPeCAl-2 was applied to minimise false positive errors by implementing pre-process masking for water and permanent bright objects and filtering ToPeCAl-2’s resultant detected fires by implementing contextual testing and cloud masking. Both ToPeCAl-2 with contextual test and ToPeCAl with cloud mask applied to Sentinel-2 provided high detection of unambiguous fire pixels (>95%) at 20 m spatial resolution. Smouldering pixels were less likely to be detected by ToPeCAl-2. The detected smouldering pixels from ToPeCAl-2 applied to Sentinel-2 with contextual testing and with cloud masking were only 35% and 56% correct, respectively; this needs further investigation and validation. These results demonstrate that even in the absence of TIR data, an adjusted ToPeCAl algorithm (ToPeCAl-2) can be applied to detect peatland fires at 20 m resolution with high accuracy especially for flaming. Overall, the implementation of ToPeCAl applied to cost-free and available Landsat-8 and Sentinel-2 data enables regular peatland fire monitoring in tropical environments at higher spatial resolution than other satellite-derived fire products.

Download Full-text

Comparison and Ground Truthing of Different Remote and Proximal Sensing Platforms to Characterize Variability in a Hedgerow-Trained Vineyard

Remote Sensing ◽

10.3390/rs13112056 ◽

2021 ◽

Vol 13 (11) ◽

pp. 2056

Author(s):

Cecilia Squeri ◽

Stefano Poni ◽

Salvatore Filippo Di Gennaro ◽

Alessandro Matese ◽

Matteo Gatti

Keyword(s):

Spatial Resolution ◽

Vegetation Index ◽

Normalized Difference Vegetation Index ◽

Growth Yield ◽

Ground Truth ◽

Large Degree ◽

Proximal Sensing ◽

Ground Truth Data ◽

Ground Truthing ◽

Sentinel 2

Appropriate characterization of intra-parcel variability is a key element for the effective application of precision farming techniques. Nowadays there are many platforms available to end users differing for pixel spatial resolution and the type of acquisition (remote or proximal). A challenging aspect pertaining to remote sensing image acquisition in the vineyard ecosystem is that, in a large majority of cases, vegetation is discontinuous and single rows alternate with strips of either bare or grassed soil. In this paper, four different satellite platforms (Sentinel-2, Spot-6, Pleiades, and WorldView-3) having different spatial resolution and MECS-VINE® proximity sensor were compared in terms of accuracy at describing spatial variability. Vineyard mapping was coupled with detailed ground truthing of growth, yield, and grape composition variables. The analysis was conducted based on vigor indices (Normalized Difference Vegetation Index or Canopy Index) and using the Moran Index (MI) to assess the degree of spatial auto-correlation for the different variables. The results obtained showed a large degree of intra-plot variability in the main agronomic parameters (pruning weight CV: 33.86%, yield: 32.09%). The univariate Moran index showed a log-linear function relating MI coefficients to the resolution levels. Comparison between vigor indices and agronomic data showed that the highest bivariate MI was reached by Pleiades followed by MECS-VINE® which also did not exhibit the negative effect of the border pixel owing to the proximal scanning acquisition. Despite WorldView-3′s high resolution (1.24 m pixel) allowing very detailed data imaging, the comparison with ground-truth data was not encouraging, probably due to the presence of pure ground pixels, while Sentinel-2 was affected by the oversized pixel at 10 m.

Download Full-text

Mineral mapping at the Ikh Shankhai porphyry Cu deposits, Mongolia using WorldView-3 data

10.5194/egusphere-egu2020-2359 ◽

2020 ◽

Author(s):

Son Youngsun ◽

Kim Kwang-Eun

Keyword(s):

Spatial Resolution ◽

Near Infrared ◽

Matched Filter ◽

Extreme Environments ◽

Ground Truth ◽

Spectral Signature ◽

Ground Truth Data ◽

Mineral Mapping ◽

Porphyry Cu ◽

Porphyry Cu Deposit

Southeastern Mongolia has limited access due to its extreme environments (long and harsh winter) and lack of infrastructure (e.g., road). Satellite remote sensing technique is one of the most effective methods to get geological information in areas where field survey is difficult. WorldView-3 (WV3), launched in August 2014, is high-spatial resolution commercial multispectral sensor developed by DigitalGlobe. WV3 measures reflected radiation in eight visible near infrared (VNIR) bands between 0.42 and 1.04 &#13211; and in eight short-wave infrared (SWIR) bands between 1.20 and 2.33, which have 1.24- and 7.5-m spatial resolution, respectively. In this study, WV3 VNIR and SWIR data were used to identify and map the various minerals in the Ikh Shankhai porphyry Cu deposit district, Mongolia.The Ikh-Shankhai porphyry Cu deposit is located within Gurvansayhan island arc terrane in southeastern (SE) Gobi mineral belt, Mongolia. The Ikh-Shankhai district include the porphyry system containing Cu-Au with primary chalcopyrite, which is classified into disseminated type and stockwork quartz type. This district consists of Late Devonian-Early Carboniferous andesite, tuff and siltstone intruded by Carboniferous-Permian granite, granodiorite and granodiorite porphyry.The WV 3 data were analyzed using mixture-tuned-matched filter (MTMF) which locates a known spectral signature in the presence of a mixed or unknown background. MTMF does not require knowledge of all of the spectral endmembers and is suited for used where materials with distinct spectral signatures occur within a single pixel. From the WV3 analysis result using mixture-tuned-matched filter (MTMF), we identified the location and abundance of alteration minerals. Advanced argillic minerals (alunite, kaolinite (or dickite), and pyrophyllite) were dominant in the lithocaps of the Budgat and Gashuun Khudag prospects; whereas, phyllic (illite) and propylitic (calcite and epidote) minerals were dominant in the areas surrounding the lithocaps. In addition, the distribution of ferric minerals (hematite and goethite) was mapped because of the oxidation of pyrite. Field work at the Ikh-Shankhai porphyry Cu district to evaluate the accuracy of the mineral mapping results was carried out in August, 2018. Reflectance spectra acquisition using a portable ASD TerraSpec Halo mineral identifier (the attached GPS covered a spectral range of 0.35 &#8211; 2.5 &#181;m) was conducted in the altered outcrops of the Ikh-Shankhai porphyry Cu district. Mineral mapping results compared well with the field spectral measurements collected for the ground truth and demonstrated WV3 capability for identifying and mapping minerals associated with hydrothermal alteration. Evaluation of the WV3 mineral mapping results using ground truth data indicates, however, a difficulty in mapping spectrally similar minerals (e.g., kaolinite and dickite) due to spectral resolution limitation.

Download Full-text

A Multispectral Bayesian Classification Method for Increased Acoustic Discrimination of Seabed Sediments Using Multi-Frequency Multibeam Backscatter Data

Geosciences ◽

10.3390/geosciences8120455 ◽

2018 ◽

Vol 8 (12) ◽

pp. 455 ◽

Cited By ~ 19

Author(s):

Timo Gaida ◽

Tengku Tengku Ali ◽

Mirjam Snellen ◽

Alireza Amiri-Simkooei ◽

Thaiënne van Dijk ◽

...

Keyword(s):

Ground Truth ◽

Single Frequency ◽

Qualitative Comparison ◽

Ground Truth Data ◽

Geological Interpretation ◽

Additional Information ◽

Triple Frequency ◽

Acoustic Discrimination ◽

Multiple Frequencies ◽

Seabed Sediments

Multi-frequency backscatter data collected from multibeam echosounders (MBESs) is increasingly becoming available. The ability to collect data at multiple frequencies at the same time is expected to allow for better discrimination between seabed sediments. We propose an extension of the Bayesian method for seabed classification to multi-frequency backscatter. By combining the information retrieved at single frequencies we produce a multispectral acoustic classification map, which allows us to distinguish more seabed environments. In this study we use three triple-frequency (100, 200, and 400 kHz) backscatter datasets acquired with an R2Sonic 2026 in the Bedford Basin, Canada in 2016 and 2017 and in the Patricia Bay, Canada in 2016. The results are threefold: (1) combining 100 and 400 kHz, in general, reveals the most additional information about the seabed; (2) the use of multiple frequencies allows for a better acoustic discrimination of seabed sediments than single-frequency data; and (3) the optimal frequency selection for acoustic sediment classification depends on the local seabed. However, a quantification of the benefit using multiple frequencies cannot clearly be determined based on the existing ground-truth data. Still, a qualitative comparison and a geological interpretation indicate an improved discrimination between different seabed environments using multi-frequency backscatter.

Download Full-text

Evaluation of TsHARP Utility for Thermal Sharpening of Sentinel-3 Satellite Images Using Sentinel-2 Visual Imagery

Remote Sensing ◽

10.3390/rs11192304 ◽

2019 ◽

Vol 11 (19) ◽

pp. 2304 ◽

Cited By ~ 3

Author(s):

Hanna Huryna ◽

Yafit Cohen ◽

Arnon Karnieli ◽

Natalya Panov ◽

William P. Kustas ◽

...

Keyword(s):

Spatial Resolution ◽

Land Surface ◽

Near Infrared ◽

Landsat 8 ◽

Large Area ◽

Primary Methods ◽

Thermal Images ◽

Spatially Distributed ◽

The Difference ◽

Sentinel 2

A spatially distributed land surface temperature is important for many studies. The recent launch of the Sentinel satellite programs paves the way for an abundance of opportunities for both large area and long-term investigations. However, the spatial resolution of Sentinel-3 thermal images is not suitable for monitoring small fragmented fields. Thermal sharpening is one of the primary methods used to obtain thermal images at finer spatial resolution at a daily revisit time. In the current study, the utility of the TsHARP method to sharpen the low resolution of Sentinel-3 thermal data was examined using Sentinel-2 visible-near infrared imagery. Compared to Landsat 8 fine thermal images, the sharpening resulted in mean absolute errors of ~1 °C, with errors increasing as the difference between the native and the target resolutions increases. Part of the error is attributed to the discrepancy between the thermal images acquired by the two platforms. Further research is due to test additional sites and conditions, and potentially additional sharpening methods, applied to the Sentinel platforms.

Download Full-text

Comparison of Surface Characteristics of the Antarctic Ice Sheet with Satellite Observations (Abstract)

Annals of Glaciology ◽

10.1017/s0260305500000975 ◽

1987 ◽

Vol 9 ◽

pp. 253

Author(s):

N. Young ◽

I. Goodwin

Keyword(s):

Surface Topography ◽

Accumulation Rate ◽

Ice Sheet ◽

Ground Truth ◽

Surface Characteristics ◽

Small Scale ◽

Landsat Images ◽

Ground Truth Data ◽

Additional Information ◽

Wilkes Land

Ground surveys of the ice sheet in Wilkes Land, Antarctica, have been made on oversnow traverses operating out of Casey. Data collected include surface elevation, accumulation rate, snow temperature, and physical characteristics of the snow cover. By the nature of the surveys, the data are mostly restricted to line profiles. In some regions, aerial surveys of surface topography have been made over a grid network. Satellite imagery and remote sensing are two means of extrapolating the results from measurements along lines to an areal presentation. They are also the only source of data over large areas of the continent. Landsat images in the visible and near infra-red wavelengths clearly depict many of the large- and small scale features of the surface. The intensity of the reflected radiation varies with the aspect and magnitude of the surface slope to reveal the surface topography. The multi-channel nature of the Landsat data is exploited to distinguish between different surface types through their different spectral signatures, e.g. bare ice, glaze, snow, etc. Additional information on surface type can be gained at a coarser scale from other satellite-borne sensors such as ESMR, SMMR, etc. Textural enhancement of the Landsat images reveals the surface micro-relief. Features in the enhanced images are compared to ground-truth data from the traverse surveys to produce a classification of surface types across the images and to determine the magnitude of the surface topography and micro-relief observed. The images can then be used to monitor changes over time.

Download Full-text

Hyperspectral remote sensing monitoring of cyanobacteria blooms in a large South American reservoir: high- and medium-spatial resolution satellite algorithm simulation

Marine and Freshwater Research ◽

10.1071/mf18429 ◽

2020 ◽

Vol 71 (5) ◽

pp. 593 ◽

Cited By ~ 3

Author(s):

A. Drozd ◽

P. de Tezanos Pinto ◽

V. Fernández ◽

M. Bazzalo ◽

F. Bordet ◽

...

Keyword(s):

Remote Sensing ◽

Spatial Resolution ◽

Near Infrared ◽

Monitoring Program ◽

Hyperspectral Remote Sensing ◽

Coefficient Of Determination ◽

South American ◽

Cell Abundance ◽

Hyperspectral Signatures ◽

Sentinel 2

We used hyperspectral remote sensing with the aim of establishing a monitoring program for cyanobacteria in a South American reservoir. We sampled at a wide temporal (2012–16; 10 seasons) and spatial (30km) gradient, and retrieved 111 field hyperspectral signatures, chlorophyll-a, cyanobacteria densities and total suspended solids. The hyperspectral signatures for cyanobacteria-dominated situations (n=75) were used to select the most suitable spectral bands in seven high- and medium-spatial resolution satellites (Sentinel 2, Landsat 5, 7 and 8, SPOT-4/5 and -6/7, WorldView 2), and for the development of chlorophyll and cyanobacteria cell abundance algorithms (λ550 – λ650+λ800) ÷ (λ550+λ650+λ800). The best-performing chlorophyll algorithm was Sentinel 2 ((λ560 – λ660+λ703) ÷ (λ560+λ660+λ703); R2=0.80), followed by WorldView 2 ((λ550 – λ660+λ720) ÷ (λ550+λ660+λ720); R2=0.78), Landsat and the SPOT series ((λ550 – λ650+λ800) ÷ (λ550+λ650+λ800); R2=0.67–0.74). When these models were run for cyanobacteria abundance, the coefficient of determination remained similar, but the root mean square error increased. This could affect the estimate of cyanobacteria cell abundance by ~20%, yet it still enable assessment of the alert level categories for risk assessment. The results of this study highlight the importance of the red and near-infrared region for identifying cyanobacteria in hypereutrophic waters, demonstrating coherence with field cyanobacteria abundance and enabling assessment of bloom distribution in this ecosystem.

Download Full-text

Fusion of Sentinel and PlanetScope images for high resolution soil moisture mapping: Algorithm and preliminary validation

10.5194/egusphere-egu2020-8324 ◽

2020 ◽

Author(s):

Mateo Gašparović ◽

Sudhir Kumar Singh

Keyword(s):

Soil Moisture ◽

High Resolution ◽

Spatial Resolution ◽

Land Degradation ◽

New Technologies ◽

Critical Role ◽

Soil Parameters ◽

Area Index ◽

Ground Truth Data ◽

Sentinel 2

High resolution remote sensing images plays a critical role in detection and monitoring of land degradation and development. Monitoring the soil parameters represents great importance for sustainable development and agriculture, as well as smart food production. The space segment component of the Copernicus programme (e.g., Sentinel-1, Sentinel-2 etc.) enables continuously monitoring of Earth&#8217;s surface at 10-m spatial resolution. New technologies, development, and minimization of sensors led to the development of micro-satellites (e.g., PlanetScope). These satellites allow us to monitor the Earth's surface daily in 3-m spatial resolution. The developed algorithm for soil moisture mapping is based on the fusion of Sentinel-2 and PlanetScope images. This allows a soil moisture mapping in high spatial resolution. Soil moisture was estimated based on the Leaf Area Index (LAI) and Enhanced Vegetation Index (EVI) using modified water cloud model. Ground-truth data were collected from 15 stations of the International Soil Moisture Network across the globe and used for mapping and validation of soil moisture. The developed algorithm provides a new knowledge that can be widely applied in various research for the detection and monitoring of land degradation and development.

Download Full-text

Deep-Learning-Based Harmonization and Super-Resolution of Near-Surface Air Temperature from CMIP6 Models (1850–2100)

10.5194/essd-2021-418 ◽

2021 ◽

Author(s):

Xikun Wei ◽

Guojie Wang ◽

Donghan Feng ◽

Zheng Duan ◽

Daniel Fiifi Tawia Hagan ◽

...

Keyword(s):

Time Series ◽

Deep Learning ◽

Spatial Resolution ◽

Bias Correction ◽

High Spatial Resolution ◽

Super Resolution ◽

Future Climate Change ◽

Observation Data ◽

Near Surface ◽

Spatial Downscaling

Abstract. Future global temperature change would have significant effects on society and ecosystems. Earth system models (ESM) are the primary tools to explore the future climate change. However, ESMs still exist great uncertainty and often run at a coarse spatial resolution (The majority of ESMs at about 2 degree). Accurate temperature data at high spatial resolution are needed to improve our understanding of the temperature variation and for many applications. We innovatively apply the deep-learning(DL) method from the Super resolution (SR) in the computer vision to merge 31 ESMs data and the proposed method can perform data merge, bias-correction and spatial-downscaling simultaneously. The SR algorithms are designed to enhance image quality and outperform much better than the traditional methods. The CRU TS (Climate Research Unit gridded Time Series) is considered as reference data in the model training process. In order to find a suitable DL method for our work, we choose five SR methodologies made by different structures. Those models are compared based on multiple evaluation metrics (Mean square error(MSE), mean absolute error(MAE) and Pearson correlation coefficient(R)) and the optimal model is selected and used to merge the monthly historical data during 1850–1900 and monthly future scenarios data (SSP1-2.6, SSP2-4.5, SSP3-7.0, SSP5-8.5) during 2015–2100 at the high spatial resolution of 0.5 degree. Results showed that the merged data have considerably improved performance than any of the individual ESM data and the ensemble mean (EM) of all ESM data in terms of both spatial and temporal aspects. The MAE displays a great improvement and the spatial distribution of the MAE become larger and larger along the latitudes in north hemisphere, presenting like a ‘tertiary class echelon’ condition. The merged product also presents excellent performance when the observation data is smooth with few fluctuations in time series. Additionally, this work proves that the DL model can be transferred to deal with the data merge, bias-correction and spatial-downscaling successfully when enough training data are available. Data can be accessed at https://doi.org/10.5281/zenodo.5746632 (Wei et al., 2021).

Download Full-text