scholarly journals On-Demand Processing of Data Cubes from Satellite Image Collections with the gdalcubes Library

Data ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. 92 ◽  
Author(s):  
Marius Appel ◽  
Edzer Pebesma
Keyword(s):  
Land Surface ◽  
Method Development ◽  
Satellite Image ◽  
Original Data ◽  
R Package ◽  
Observation Data ◽  
Data Cubes ◽  
On Demand ◽  
Temperature And Precipitation ◽  
Data Products

Earth observation data cubes are increasingly used as a data structure to make large collections of satellite images easily accessible to scientists. They hide complexities in the data such that data users can concentrate on the analysis rather than on data management. However, the construction of data cubes is not trivial and involves decisions that must be taken with regard to any particular analyses. This paper proposes on-demand data cubes, which are constructed on the fly when data users process the data. We introduce the open-source C++ library and R package gdalcubes for the construction and processing of on-demand data cubes from satellite image collections, and show how it supports interactive method development workflows where data users can initially try methods on small subsamples before running analyses on high resolution and/or large areas. Two study cases, one on processing Sentinel-2 time series and the other on combining vegetation, land surface temperature, and precipitation data, demonstrate and evaluate this implementation. While results suggest that on-demand data cubes implemented in gdalcubes support interactivity and allow for combining multiple data products, the speed-up effect also strongly depends on how original data products are organized. The potential for cloud deployment is discussed.

Evaluating model-simulated and satellite-derived SM using in situ observations under different environment conditions in China

2020 ◽  
Author(s):  
Yuanyuan Wang ◽  
Guicai Li
Keyword(s):  
Land Surface ◽  
Large Scale ◽  
Microwave Remote Sensing ◽  
Ground Truth ◽  
Original Data ◽  
Observation Data ◽  
Monitoring Networks ◽  
The Future ◽  
In Situ Observations

<p>Soil moisture (SM) is a key variable in understanding the climate system through its controls on the land surface energy and water budget. Large scale SM products have become increasingly available thanks to development in microwave remote sensing and land surface modeling. Comprehensive assessments on the reliability of satellite-derived and model-simulated SM products are essential for their improvement and application. In this research, the active, passive and combined Climate Change Initiative (CCI V04.2) SM products and the China Land Data Assimilation System (CLDAS V2.0) SM products were evaluated by comparing with in situ observed data over three networks in China: Hebi, Naqu and Heihe. The three sites have different environmental conditions and sensor densities, providing observations covering more than 2 years. Four statistic scores were calculated: <em>R</em> (considering both original data and anomalies), <em>Bias</em>, <em>RMSE</em>, <em>ubRMSE</em>. TC (Triple Collocation) analysis was also carried out in which uncertainties in observations are taken into account. Results indicate that the performance of the two SM products varies between the monitoring networks. For Naqu site, both products show good performance, with CCI-SM showing slightly higher <em>R</em> and lower <em>ubRMSE</em>. For Hebi site, CLDAS-SM performs better than CCI-SM, whereas for Heihe site, CLDAS-SM performs worse than CCI-SM. The expected uncertainty (0.04 m<sup>3</sup>/m<sup>3</sup>) can be achieved in Naqu and Heihe site by CCI-SM, and in Hebi and Naqu site by CLDAS-SM, which is quite encouraging. The two products agree in terms of sign of the <em>Bias</em> value, which is positive in Hebi and negative in Naqu and Heihe. Among all the three networks, Heihe site exhibits the lowest accuracy due to its complicated terrain and heterogeneous land surface.<em> R<sub>anom</sub></em> of CLDAS-SM in Heihe is close to 0, indicating its inability to capture short term variability. TC results reveal that for Naqu site the observation data have quite good qualities, while for Hebi site CLDAS-SM is more approximate to ‘ground truth’ than in situ observations, suggesting a refinement of network maybe needed in the future. Overall, the two products are complementary. CLDAS-SM performs better in populated area (e.g., Hebi) where meteorological forcing is more accurate and CCI-SM performs better in remote areas (Naqu, Heihe) where RFI is usually low. More reliable validation networks are needed in the future to comprehensively understand the advantages and disadvantage of the two SM products in China.</p>

scholarly journals A permutation test to probe the statistical significance of sudden gain frequency: An application to patterns of change in cognitive and interpersonal therapy for depression

2020 ◽  
Author(s):  
Lorenzo Lorenzo-Luaces ◽  
Milan Wiedemann ◽  
M.J.H. Huibers ◽  
Lotte H.J.M. Lemmens
Keyword(s):  
Method Development ◽  
Permutation Test ◽  
Statistical Significance ◽  
Original Data ◽  
R Package ◽  
Chance Level ◽  
P Value ◽  
Interpersonal Therapy ◽  
Sudden Gains ◽  
Positive Treatment

Objectives: Sudden gains were first identified in cognitive therapy (CT) for depression to characterize large and stable improvements. They have subsequently been studied in at least 50 studies across a range of disorders and treatments. There is evidence that sudden gains are reliably associated with positive treatment outcomes. Nonetheless, simulations have suggested that sudden gains may occur spuriously. We propose the use of a permutation test to yield a p-value to ascertain whether sudden gains occur above and beyond the level that can be expected by chance in a given sample. Methods: We reanalyzed the study by Lemmens et al. (2016) to explore the utility of a permutation test for the occurrence and effects of sudden gains. We permuted session-by-session depression scores within each session before identifying sudden gains to explore the effects of the number of identified sudden gains. We also permuted sudden gains status. For both analyses we resampled 10,000 random datasets. Results: In the permuted samples, the mean number of identified sudden gains was 29.43, replicating prior work that suggests caution about the occurrence of sudden gains. However, this rate was lower than the number of gains (n = 52) occurring in the original data (p < 0.001). The association between sudden gains and outcomes was also above and beyond chance level (p < 0.001).Discussion: These data suggest that sudden gains occur above and beyond chance level. We provide code for these analyses and updated the suddengains R package to facilitate replications and further method development.

scholarly journals End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++

2019 ◽  
Vol 11 (11) ◽  
pp. 1382 ◽  
Author(s):  
Daifeng Peng ◽  
Yongjun Zhang ◽  
Haiyan Guan
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Change Detection ◽  
Land Surface ◽  
Satellite Image ◽  
Feature Representation ◽  
Observation Data ◽  
Feature Maps ◽  
High Resolution Satellite Images ◽  
End To End

Change detection (CD) is essential to the accurate understanding of land surface changes using available Earth observation data. Due to the great advantages in deep feature representation and nonlinear problem modeling, deep learning is becoming increasingly popular to solve CD tasks in remote-sensing community. However, most existing deep learning-based CD methods are implemented by either generating difference images using deep features or learning change relations between pixel patches, which leads to error accumulation problems since many intermediate processing steps are needed to obtain final change maps. To address the above-mentioned issues, a novel end-to-end CD method is proposed based on an effective encoder-decoder architecture for semantic segmentation named UNet++, where change maps could be learned from scratch using available annotated datasets. Firstly, co-registered image pairs are concatenated as an input for the improved UNet++ network, where both global and fine-grained information can be utilized to generate feature maps with high spatial accuracy. Then, the fusion strategy of multiple side outputs is adopted to combine change maps from different semantic levels, thereby generating a final change map with high accuracy. The effectiveness and reliability of our proposed CD method are verified on very-high-resolution (VHR) satellite image datasets. Extensive experimental results have shown that our proposed approach outperforms the other state-of-the-art CD methods.

scholarly journals Hydrostreamer v1.0 – improved streamflow predictions for local applications from an ensemble of downscaled global runoff products

2021 ◽  
Vol 14 (8) ◽  
pp. 5155-5181
Author(s):  
Marko Kallio ◽  
Joseph H. A. Guillaume ◽  
Vili Virkki ◽  
Matti Kummu ◽  
Kirsi Virrantaus
Keyword(s):  
Land Surface ◽  
Model Averaging ◽  
R Package ◽  
Observation Data ◽  
Areal Interpolation ◽  
Multiple Input ◽  
Minimal Data ◽  
Intercomparison Project ◽  
Improved Performance ◽  
Area Unit

Abstract. An increasing number of different types of hydrological, land surface, and rainfall–runoff models exist to estimate streamflow in river networks. Results from various model runs from global to local scales are readily available online. However, the usability of these products is often limited, as they often come aggregated in spatial units which are not compatible with the desired analysis purpose. We present here an R package, a software library Hydrostreamer v1.0, which aims to improve the usability of existing runoff products by addressing the modifiable area unit problem and allows non-experts with little knowledge of hydrology-specific modelling issues and methods to use them for their analyses. Hydrostreamer workflow includes (1) interpolation from source zones to target zones, (2) river routing, and (3) data assimilation via model averaging, given multiple input runoff and observation data. The software implements advanced areal interpolation methods and area-to-line interpolation not available in other products and is the first R package to provide vector-based routing. Hydrostreamer is kept as simple as possible – intuitive with minimal data requirements – and minimises the need for calibration. We tested the performance of Hydrostreamer by downscaling freely available coarse-resolution global runoff products from the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) in an application in 3S Basin in Southeast Asia. Results are compared to observed discharges as well as two benchmark streamflow data products, finding comparable or improved performance. Hydrostreamer v1.0 is open source and is available from http://github.com/mkkallio/hydrostreamer/ (last access: 5 May 2021) under the MIT licence.

scholarly journals Hydrostreamer v1.0 – improved streamflow predictions for local applications from an ensemble of downscaled global runoff products

2020 ◽  
Author(s):  
Marko Kallio ◽  
Joseph H. A. Guillaume ◽  
Vili Virkki ◽  
Matti Kummu ◽  
Kirsi Virrantaus
Keyword(s):  
Land Surface ◽  
Model Averaging ◽  
R Package ◽  
Observation Data ◽  
Areal Interpolation ◽  
Multiple Input ◽  
Minimal Data ◽  
Intercomparison Project ◽  
Improved Performance ◽  
Area Unit

Abstract. An increasing number of different types of hydrological, land surface, and rainfall-runoff models exist to estimate streamflow in river networks. Results from various model runs from global to local scale are readily available online. However, the usability of these products is often limited, as they often come aggregated in spatial units which are not compatible with the desired analysis purpose. We present here an R package, a software library hydrostreamer v1.0 which aims to improve the usability of existing runoff products by addressing the Modifiable Area Unit Problem, and allows non-experts with little knowledge of hydrology-specific modelling issues and methods to use them for their analyses. Hydrostreamer workflow includes 1) interpolation from source zones to target zones, 2) river routing, and 3) data assimilation via model averaging, given multiple input runoff and observation data. The software implements advanced areal interpolation methods and area-to-line interpolation not available in other products, and is the first R package to provide vector-based routing. Hydrostreamer is kept as simple as possible – intuitive with minimal data requirements – and minimizes need for calibration. We tested the performance of hydrostreamer by downscaling freely available coarse-resolution global runoff products from the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) in an application in 3S Basin in Southeast Asia. Results are compared to observed discharges as well as two benchmark streamflow data products, finding comparable or improved performance. Hydrostreamer v1.0 is open source and is available from http://github.com/mkkallio/hydrostreamer/ under MIT licence.

scholarly journals ACCESSING AND PROCESSING BRAZILIAN EARTH OBSERVATION DATA CUBES WITH THE OPEN DATA CUBE PLATFORM

2021 ◽  
Vol V-4-2021 ◽  
pp. 153-159
Author(s):  
V. C. F. Gomes ◽  
F. M. Carlos ◽  
G. R. Queiroz ◽  
K. R. Ferreira ◽  
R. Santos
Keyword(s):  
Open Source ◽  
Open Data ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Functional Modules ◽  
Data Cubes ◽  
Earth Observations ◽  
Data Products ◽  
Earth Observation Data

Abstract. Recently, several technologies have emerged to address the need to process and analyze large volumes of Earth Observations (EO) data. The concept of Earth Observations Data Cubes (EODC) appears, in this context, as the paradigm of technologies that aim to structure and facilitate the way users handle this type of data. Some projects have adopted this concept in developing their technologies, such as the Open Data Cube (ODC) framework and the Brazil Data Cube (BDC) platform, which provide open-source tools capable of managing, processing, analyzing, and disseminating EO data. This work presents an approach to integrate these technologies through the access and processing of data products from the BDC platform in the ODC framework. For this, we developed a tool to automate the process of searching, converting, and indexing data between these two systems. Besides, four ODC functional modules have been customized to work with BDC data. The tool developed and the changes made to the ODC modules expand the potential for other initiatives to take advantage of the features available in the ODC.

scholarly journals Quantifying Tree Cover Loss in Urban Forests within Nairobi City Metropolitan Area from Earth Observation Data

2020 ◽  
Vol 3 (1) ◽  
pp. 78
Author(s):  
Francis Oloo ◽  
Godwin Murithi ◽  
Charlynne Jepkosgei
Keyword(s):  
Land Cover ◽  
Metropolitan Area ◽  
Land Surface ◽  
Urban Forest ◽  
Urban Forests ◽  
Earth Observation ◽  
Tree Cover ◽  
Observation Data ◽  
Vegetation Health ◽  
Earth Observation Data

Urban forests contribute significantly to the ecological integrity of urban areas and the quality of life of urban dwellers through air quality control, energy conservation, improving urban hydrology, and regulation of land surface temperatures (LST). However, urban forests are under threat due to human activities, natural calamities, and bioinvasion continually decimating forest cover. Few studies have used fine-scaled Earth observation data to understand the dynamics of tree cover loss in urban forests and the sustainability of such forests in the face of increasing urban population. The aim of this work was to quantify the spatial and temporal changes in urban forest characteristics and to assess the potential drivers of such changes. We used data on tree cover, normalized difference vegetation index (NDVI), and land cover change to quantify tree cover loss and changes in vegetation health in urban forests within the Nairobi metropolitan area in Kenya. We also used land cover data to visualize the potential link between tree cover loss and changes in land use characteristics. From approximately 6600 hectares (ha) of forest land, 720 ha have been lost between 2000 and 2019, representing about 11% loss in 20 years. In six of the urban forests, the trend of loss was positive, indicating a continuing disturbance of urban forests around Nairobi. Conversely, there was a negative trend in the annual mean NDVI values for each of the forests, indicating a potential deterioration of the vegetation health in the forests. A preliminary, visual inspection of high-resolution imagery in sample areas of tree cover loss showed that the main drivers of loss are the conversion of forest lands to residential areas and farmlands, implementation of big infrastructure projects that pass through the forests, and extraction of timber and other resources to support urban developments. The outcome of this study reveals the value of Earth observation data in monitoring urban forest resources.

scholarly journals Using Long-Term Earth Observation Data to Reveal the Factors Contributing to the Early 2020 Desert Locust Upsurge and the Resulting Vegetation Loss

2021 ◽  
Vol 13 (4) ◽  
pp. 680
Author(s):  
Lei Wang ◽  
Wen Zhuo ◽  
Zhifang Pei ◽  
Xingyuan Tong ◽  
Wei Han ◽  
...  
Keyword(s):  
Soil Moisture ◽  
Land Surface ◽  
Arabian Peninsula ◽  
Vegetation Coverage ◽  
Observation Data ◽  
Desert Locust ◽  
Meteorological Observations ◽  
Earth Observation Data ◽  
Vegetation Loss

Massive desert locust swarms have been threatening and devouring natural vegetation and agricultural crops in East Africa and West Asia since 2019, and the event developed into a rare and globally concerning locust upsurge in early 2020. The breeding, maturation, concentration and migration of locusts rely on appropriate environmental factors, mainly precipitation, temperature, vegetation coverage and land-surface soil moisture. Remotely sensed images and long-term meteorological observations across the desert locust invasion area were analyzed to explore the complex drivers, vegetation losses and growing trends during the locust upsurge in this study. The results revealed that (1) the intense precipitation events in the Arabian Peninsula during 2018 provided suitable soil moisture and lush vegetation, thus promoting locust breeding, multiplication and gregarization; (2) the regions affected by the heavy rainfall in 2019 shifted from the Arabian Peninsula to West Asia and Northeast Africa, thus driving the vast locust swarms migrating into those regions and causing enormous vegetation loss; (3) the soil moisture and NDVI anomalies corresponded well with the locust swarm movements; and (4) there was a low chance the eastwardly migrating locust swarms would fly into the Indochina Peninsula and Southwest China.

scholarly journals Fundamental Climate Data Records of Microwave Brightness Temperatures

2018 ◽  
Vol 10 (8) ◽  
pp. 1306 ◽  
Author(s):  
Wesley Berg ◽  
Rachael Kroodsma ◽  
Christian Kummerow ◽  
Darren McKague
Keyword(s):  
Land Surface ◽  
Surface Wind ◽  
Precipitable Water ◽  
Original Data ◽  
Climate Data ◽  
Sea Ice Extent ◽  
Brightness Temperatures ◽  
Precipitation Total ◽  
Data Record ◽  
Microwave Imager

An intercalibrated Fundamental Climate Data Record (FCDR) of brightness temperatures (Tb) has been developed using data from a total of 14 research and operational conical-scanning microwave imagers. This dataset provides a consistent 30+ year data record of global observations that is well suited for retrieving estimates of precipitation, total precipitable water, cloud liquid water, ocean surface wind speed, sea ice extent and concentration, snow cover, soil moisture, and land surface emissivity. An initial FCDR was developed for a series of ten Special Sensor Microwave/Imager (SSM/I) and Special Sensor Microwave Imager Sounder (SSMIS) instruments on board the Defense Meteorological Satellite Program spacecraft. An updated version of this dataset, including additional NASA and Japanese sensors, has been developed as part of the Global Precipitation Measurement (GPM) mission. The FCDR development efforts involved quality control of the original data, geolocation corrections, calibration corrections to account for cross-track and time-dependent calibration errors, and intercalibration to ensure consistency with the calibration reference. Both the initial SSMI(S) and subsequent GPM Level 1C FCDR datasets are documented, updated in near real-time, and publicly distributed.

Sign in / Sign up

Export Citation Format

Share Document