ACCESSING AND PROCESSING BRAZILIAN EARTH OBSERVATION DATA CUBES WITH THE OPEN DATA CUBE PLATFORM

Abstract. Recently, several technologies have emerged to address the need to process and analyze large volumes of Earth Observations (EO) data. The concept of Earth Observations Data Cubes (EODC) appears, in this context, as the paradigm of technologies that aim to structure and facilitate the way users handle this type of data. Some projects have adopted this concept in developing their technologies, such as the Open Data Cube (ODC) framework and the Brazil Data Cube (BDC) platform, which provide open-source tools capable of managing, processing, analyzing, and disseminating EO data. This work presents an approach to integrate these technologies through the access and processing of data products from the BDC platform in the ODC framework. For this, we developed a tool to automate the process of searching, converting, and indexing data between these two systems. Besides, four ODC functional modules have been customized to work with BDC data. The tool developed and the changes made to the ODC modules expand the potential for other initiatives to take advantage of the features available in the ODC.

Download Full-text

Achieving the Full Vision of Earth Observation Data Cubes

Data ◽

10.3390/data4030094 ◽

2019 ◽

Vol 4 (3) ◽

pp. 94 ◽

Cited By ~ 6

Author(s):

Steve Kopp ◽

Peter Becker ◽

Abhijit Doshi ◽

Dawn J. Wright ◽

Kaixi Zhang ◽

...

Keyword(s):

Image Data ◽

Earth Observation ◽

Data Cube ◽

Observation Data ◽

Data Cubes ◽

Multiple Data ◽

Storage Location ◽

Science Community ◽

Earth Observation Data ◽

And Storage

Earth observation imagery have traditionally been expensive, difficult to find and access, and required specialized skills and software to transform imagery into actionable information. This has limited adoption by the broader science community. Changes in cost of imagery and changes in computing technology over the last decade have enabled a new approach for how to organize, analyze, and share Earth observation imagery, broadly referred to as a data cube. The vision and promise of image data cubes is to lower these hurdles and expand the user community by making analysis ready data readily accessible and providing modern approaches to more easily analyze and visualize the data, empowering a larger community of users to improve their knowledge of place and make better informed decisions. Image data cubes are large collections of temporal, multivariate datasets typically consisting of analysis ready multispectral Earth observation data. Several flavors and variations of data cubes have emerged. To simplify access for end users we developed a flexible approach supporting multiple data cube styles, referencing images in their existing structure and storage location, enabling fast access, visualization, and analysis from a wide variety of web and desktop applications. We provide here an overview of that approach and three case studies.

Download Full-text

Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes

Data ◽

10.3390/data4040147 ◽

2019 ◽

Vol 4 (4) ◽

pp. 147 ◽

Cited By ~ 9

Author(s):

Gregory Giuliani ◽

Gilberto Camara ◽

Brian Killough ◽

Stuart Minchin

Keyword(s):

Open Science ◽

Earth Observation ◽

Data Cube ◽

Observation Data ◽

Special Issue ◽

Data Repositories ◽

Data Cubes ◽

Information Power ◽

Wide Range ◽

Earth Observation Data

Earth Observation Data Cubes (EODC) have emerged as a promising solution to efficiently and effectively handle Big Earth Observation (EO) Data generated by satellites and made freely and openly available from different data repositories. The aim of this Special Issue, “Earth Observation Data Cube”, in Data, is to present the latest advances in EODC development and implementation, including innovative approaches for the exploitation of satellite EO data using multi-dimensional (e.g., spatial, temporal, spectral) approaches. This Special Issue contains 14 articles covering a wide range of topics such as Synthetic Aperture Radar (SAR), Analysis Ready Data (ARD), interoperability, thematic applications (e.g., land cover, snow cover mapping), capacity development, semantics, processing techniques, as well as national implementations and best practices. These papers made significant contributions to the advancement of a more Open and Reproducible Earth Observation Science, reducing the gap between users’ expectations for decision-ready products and current Big Data analytical capabilities, and ultimately unlocking the information power of EO data by transforming them into actionable knowledge.

Download Full-text

Semantic Earth Observation Data Cubes

Data ◽

10.3390/data4030102 ◽

2019 ◽

Vol 4 (3) ◽

pp. 102 ◽

Cited By ~ 5

Author(s):

Hannah Augustin ◽

Martin Sudmanns ◽

Dirk Tiede ◽

Stefan Lang ◽

Andrea Baraldi

Keyword(s):

Data Retrieval ◽

Earth Observation ◽

Data Cube ◽

Observation Data ◽

Data Cubes ◽

Sensory Data ◽

Main Challenge ◽

Data Content ◽

Spatio Temporal ◽

Earth Observation Data

There is an increasing amount of free and open Earth observation (EO) data, yet more information is not necessarily being generated from them at the same rate despite high information potential. The main challenge in the big EO analysis domain is producing information from EO data, because numerical, sensory data have no semantic meaning; they lack semantics. We are introducing the concept of a semantic EO data cube as an advancement of state-of-the-art EO data cubes. We define a semantic EO data cube as a spatio-temporal data cube containing EO data, where for each observation at least one nominal (i.e., categorical) interpretation is available and can be queried in the same instance. Here we clarify and share our definition of semantic EO data cubes, demonstrating how they enable different possibilities for data retrieval, semantic queries based on EO data content and semantically enabled analysis. Semantic EO data cubes are the foundation for EO data expert systems, where new information can be inferred automatically in a machine-based way using semantic queries that humans understand. We argue that semantic EO data cubes are better positioned to handle current and upcoming big EO data challenges than non-semantic EO data cubes, while facilitating an ever-diversifying user-base to produce their own information and harness the immense potential of big EO data.

Download Full-text

Mapping slums and model population density using earth observation data and open source solutions

2019 Joint Urban Remote Sensing Event (JURSE) ◽

10.1109/jurse.2019.8808934 ◽

2019 ◽

Author(s):

Tais Grippa ◽

Stefanos Georganos ◽

Sabine Vanhuysse ◽

Moritz Lennert ◽

Nicholus Mboga ◽

...

Keyword(s):

Population Density ◽

Open Source ◽

Earth Observation ◽

Observation Data ◽

Model Population ◽

Earth Observation Data

Download Full-text

Analyzing large-scale Earth Observation data repositories made simple with OpenEO Platform

10.5194/egusphere-egu21-9602 ◽

2021 ◽

Author(s):

Edzer Pebesma ◽

Patrick Griffiths ◽

Christian Briese ◽

Alexander Jacob ◽

Anze Skerlevaj ◽

...

Keyword(s):

Large Scale ◽

Virtual Machines ◽

Google Earth ◽

Earth Observation ◽

Data Cube ◽

Observation Data ◽

Data Repositories ◽

Software Ecosystem ◽

Wide Range ◽

Earth Observation Data

The OpenEO API allows the analysis of large amounts of Earth Observation data using a high-level abstraction of data and processes. Rather than focusing on the management of virtual machines and millions of imagery files, it allows to create jobs that take a spatio-temporal section of an image collection (such as Sentinel L2A), and treat it as a data cube. Processes iterate or aggregate over pixels, spatial areas, spectral bands, or time series, while working at arbitrary spatial resolution. This pattern, pioneered by Google Earth Engine&#8482; (GEE), lets the user focus on the science rather than on data management.The openEO H2020 project (2017-2020) has developed the API as well as an ecosystem of software around it, including clients (JavaScript, Python, R, QGIS, browser-based), back-ends that translate API calls into existing image analysis or GIS software or services (for Sentinel Hub, WCPS, Open Data Cube, GRASS GIS, GeoTrellis/GeoPySpark, and GEE) as well as a hub that allows querying and searching openEO providers for their capabilities and datasets. The project demonstrated this software in a number of use cases, where identical processing instructions were sent to different implementations, allowing comparison of returned results.A follow-up, ESA-funded project &#8220;openEO Platform&#8221; realizes the API and progresses the software ecosystem into operational services and applications that are accessible to everyone, that involve federated deployment (using the clouds managed by EODC, Terrascope, CreoDIAS and EuroDataCube), that will provide payment models (&#8220;pay per compute job&#8221;) conceived and implemented following the user community needs and that will use the EOSC (European Open Science Cloud) marketplace for dissemination and authentication. A wide range of large-scale cases studies will demonstrate the ability of the openEO Platform to scale to large data volumes.&#160; The case studies to be addressed include on-demand ARD generation for SAR and multi-spectral data, agricultural demonstrators like crop type and condition monitoring, forestry services like near real time forest damage assessment as well as canopy cover mapping, environmental hazard monitoring of floods and air pollution as well as security applications in terms of vessel detection in the mediterranean sea.While the landscape of cloud-based EO platforms and services has matured and diversified over the past decade, we believe there are strong advantages for scientists and government agencies to adopt the openEO approach. Beyond the absence of vendor/platform lock-in or EULA&#8217;s we mention the abilities to (i) run arbitrary user code (e.g. written in R or Python) close to the data, (ii) carry out scientific computations on an entirely open source software stack, (iii) integrate different platforms (e.g., different cloud providers offering different datasets), and (iv) help create and extend this software ecosystem. openEO uses the OpenAPI standard, aligns with modern OGC API standards, and uses the STAC (SpatioTemporal Asset Catalog) to describe image collections and image tiles.

Download Full-text

Data Cube on Demand (DCoD): Generating an earth observation Data Cube anywhere in the world

International Journal of Applied Earth Observation and Geoinformation ◽

10.1016/j.jag.2019.102035 ◽

2020 ◽

Vol 87 ◽

pp. 102035 ◽

Cited By ~ 8

Author(s):

Gregory Giuliani ◽

Bruno Chatenoux ◽

Thomas Piller ◽

Frédéric Moser ◽

Pierre Lacroix

Keyword(s):

Earth Observation ◽

Data Cube ◽

Observation Data ◽

On Demand ◽

The World ◽

Earth Observation Data

Download Full-text

Remote Sensing

Big Data in Bioeconomy ◽

10.1007/978-3-030-71069-9_4 ◽

2021 ◽

pp. 49-61

Author(s):

Miguel Ángel Esbrí

Keyword(s):

Remote Sensing ◽

Big Data ◽

Information Access ◽

Open Data ◽

Earth Observation ◽

Observation Data ◽

Related Data ◽

Data Formats ◽

Earth Observation Data ◽

Access Services

AbstractIn this chapter we present the concepts of remote sensing and Earth Observation and, explain why several of their characteristics (volume, variety and velocity) make us consider Earth Observation as Big Data. Thereafter, we discuss the most commonly open data formats used to store and share the data. The main sources of Earth Observation data are also described, with particular focus on the constellation of Sentinel satellites, Copernicus Hub and its six thematic services, as well as other private initiatives like the five Copernicus-related Data and Information Access Services and Sentinel Hub. Next, we present an overview of representative software technologies for efficiently describing, storing, querying and accessing Earth Observation datasets. The chapter concludes with a summary of the Earth Observation datasets used in each DataBio pilot.

Download Full-text