Analyzing large-scale Earth Observation data repositories made simple with OpenEO Platform

2021 ◽  
Author(s):  
Edzer Pebesma ◽  
Patrick Griffiths ◽  
Christian Briese ◽  
Alexander Jacob ◽  
Anze Skerlevaj ◽  
...  
Keyword(s):  
Large Scale ◽  
Virtual Machines ◽  
Google Earth ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Repositories ◽  
Software Ecosystem ◽  
Wide Range ◽  
Earth Observation Data

<p>The OpenEO API allows the analysis of large amounts of Earth Observation data using a high-level abstraction of data and processes. Rather than focusing on the management of virtual machines and millions of imagery files, it allows to create jobs that take a spatio-temporal section of an image collection (such as Sentinel L2A), and treat it as a data cube. Processes iterate or aggregate over pixels, spatial areas, spectral bands, or time series, while working at arbitrary spatial resolution. This pattern, pioneered by Google Earth Engine™ (GEE), lets the user focus on the science rather than on data management.</p><p>The openEO H2020 project (2017-2020) has developed the API as well as an ecosystem of software around it, including clients (JavaScript, Python, R, QGIS, browser-based), back-ends that translate API calls into existing image analysis or GIS software or services (for Sentinel Hub, WCPS, Open Data Cube, GRASS GIS, GeoTrellis/GeoPySpark, and GEE) as well as a hub that allows querying and searching openEO providers for their capabilities and datasets. The project demonstrated this software in a number of use cases, where identical processing instructions were sent to different implementations, allowing comparison of returned results.</p><p>A follow-up, ESA-funded project “openEO Platform” realizes the API and progresses the software ecosystem into operational services and applications that are accessible to everyone, that involve federated deployment (using the clouds managed by EODC, Terrascope, CreoDIAS and EuroDataCube), that will provide payment models (“pay per compute job”) conceived and implemented following the user community needs and that will use the EOSC (European Open Science Cloud) marketplace for dissemination and authentication. A wide range of large-scale cases studies will demonstrate the ability of the openEO Platform to scale to large data volumes.  The case studies to be addressed include on-demand ARD generation for SAR and multi-spectral data, agricultural demonstrators like crop type and condition monitoring, forestry services like near real time forest damage assessment as well as canopy cover mapping, environmental hazard monitoring of floods and air pollution as well as security applications in terms of vessel detection in the mediterranean sea.</p><p>While the landscape of cloud-based EO platforms and services has matured and diversified over the past decade, we believe there are strong advantages for scientists and government agencies to adopt the openEO approach. Beyond the absence of vendor/platform lock-in or EULA’s we mention the abilities to (i) run arbitrary user code (e.g. written in R or Python) close to the data, (ii) carry out scientific computations on an entirely open source software stack, (iii) integrate different platforms (e.g., different cloud providers offering different datasets), and (iv) help create and extend this software ecosystem. openEO uses the OpenAPI standard, aligns with modern OGC API standards, and uses the STAC (SpatioTemporal Asset Catalog) to describe image collections and image tiles.</p>

scholarly journals Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes

Data ◽  
2019 ◽  
Vol 4 (4) ◽  
pp. 147 ◽  
Author(s):  
Gregory Giuliani ◽  
Gilberto Camara ◽  
Brian Killough ◽  
Stuart Minchin
Keyword(s):  
Open Science ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Special Issue ◽  
Data Repositories ◽  
Data Cubes ◽  
Information Power ◽  
Wide Range ◽  
Earth Observation Data

Earth Observation Data Cubes (EODC) have emerged as a promising solution to efficiently and effectively handle Big Earth Observation (EO) Data generated by satellites and made freely and openly available from different data repositories. The aim of this Special Issue, “Earth Observation Data Cube”, in Data, is to present the latest advances in EODC development and implementation, including innovative approaches for the exploitation of satellite EO data using multi-dimensional (e.g., spatial, temporal, spectral) approaches. This Special Issue contains 14 articles covering a wide range of topics such as Synthetic Aperture Radar (SAR), Analysis Ready Data (ARD), interoperability, thematic applications (e.g., land cover, snow cover mapping), capacity development, semantics, processing techniques, as well as national implementations and best practices. These papers made significant contributions to the advancement of a more Open and Reproducible Earth Observation Science, reducing the gap between users’ expectations for decision-ready products and current Big Data analytical capabilities, and ultimately unlocking the information power of EO data by transforming them into actionable knowledge.

scholarly journals An Overview of Platforms for Big Earth Observation Data Management and Analysis

2020 ◽  
Vol 12 (8) ◽  
pp. 1253 ◽  
Author(s):  
Vitor Gomes ◽  
Gilberto Queiroz ◽  
Karine Ferreira
Keyword(s):  
Data Management ◽  
Data Access ◽  
Database Systems ◽  
Google Earth ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Novel Technologies ◽  
Community Interest ◽  
Earth Observation Data

In recent years, Earth observation (EO) satellites have generated big amounts of geospatial data that are freely available for society and researchers. This scenario brings challenges for traditional spatial data infrastructures (SDI) to properly store, process, disseminate and analyze these big data sets. To meet these demands, novel technologies have been proposed and developed, based on cloud computing and distributed systems, such as array database systems, MapReduce systems and web services to access and process big Earth observation data. Currently, these technologies have been integrated into cutting edge platforms in order to support a new generation of SDI for big Earth observation data. This paper presents an overview of seven platforms for big Earth observation data management and analysis—Google Earth Engine (GEE), Sentinel Hub, Open Data Cube (ODC), System for Earth Observation Data Access, Processing and Analysis for Land Monitoring (SEPAL), openEO, JEODPP, and pipsCloud. We also provide a comparison of these platforms according to criteria that represent capabilities of the EO community interest.

scholarly journals FREE GLOBAL DSM ASSESSMENT ON LARGE SCALE AREAS EXPLOITING THE POTENTIALITIES OF THE INNOVATIVE GOOGLE EARTH ENGINE PLATFORM

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 627-633 ◽  
Author(s):  
A. Nascetti ◽  
M. Di Rita ◽  
R. Ravanelli ◽  
M. Amicuzi ◽  
S. Esposito ◽  
...  
Keyword(s):  
High Performance ◽  
Large Scale ◽  
Global Scale ◽  
Google Earth ◽  
Observation Data ◽  
Geometric Accuracy ◽  
Computing Platform ◽  
Aster Gdem ◽  
Google Earth Engine ◽  
Earth Observation Data

The high-performance cloud-computing platform Google Earth Engine has been developed for global-scale analysis based on the Earth observation data. In particular, in this work, the geometric accuracy of the two most used nearly-global free DSMs (SRTM and ASTER) has been evaluated on the territories of four American States (Colorado, Michigan, Nevada, Utah) and one Italian Region (Trentino Alto- Adige, Northern Italy) exploiting the potentiality of this platform. These are large areas characterized by different terrain morphology, land covers and slopes. The assessment has been performed using two different reference DSMs: the USGS National Elevation Dataset (NED) and a LiDAR acquisition. The DSMs accuracy has been evaluated through computation of standard statistic parameters, both at global scale (considering the whole State/Region) and in function of the terrain morphology using several slope classes. The geometric accuracy in terms of Standard deviation and NMAD, for SRTM range from 2-3 meters in the first slope class to about 45 meters in the last one, whereas for ASTER, the values range from 5-6 to 30 meters.<br><br> In general, the performed analysis shows a better accuracy for the SRTM in the flat areas whereas the ASTER GDEM is more reliable in the steep areas, where the slopes increase. These preliminary results highlight the GEE potentialities to perform DSM assessment on a global scale.

scholarly journals Achieving the Full Vision of Earth Observation Data Cubes

Data ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. 94 ◽  
Author(s):  
Steve Kopp ◽  
Peter Becker ◽  
Abhijit Doshi ◽  
Dawn J. Wright ◽  
Kaixi Zhang ◽  
...  
Keyword(s):  
Image Data ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Cubes ◽  
Multiple Data ◽  
Storage Location ◽  
Science Community ◽  
Earth Observation Data ◽  
And Storage

Earth observation imagery have traditionally been expensive, difficult to find and access, and required specialized skills and software to transform imagery into actionable information. This has limited adoption by the broader science community. Changes in cost of imagery and changes in computing technology over the last decade have enabled a new approach for how to organize, analyze, and share Earth observation imagery, broadly referred to as a data cube. The vision and promise of image data cubes is to lower these hurdles and expand the user community by making analysis ready data readily accessible and providing modern approaches to more easily analyze and visualize the data, empowering a larger community of users to improve their knowledge of place and make better informed decisions. Image data cubes are large collections of temporal, multivariate datasets typically consisting of analysis ready multispectral Earth observation data. Several flavors and variations of data cubes have emerged. To simplify access for end users we developed a flexible approach supporting multiple data cube styles, referencing images in their existing structure and storage location, enabling fast access, visualization, and analysis from a wide variety of web and desktop applications. We provide here an overview of that approach and three case studies.

scholarly journals Data Cube on Demand (DCoD): Generating an earth observation Data Cube anywhere in the world

2020 ◽  
Vol 87 ◽  
pp. 102035 ◽  
Author(s):  
Gregory Giuliani ◽  
Bruno Chatenoux ◽  
Thomas Piller ◽  
Frédéric Moser ◽  
Pierre Lacroix
Keyword(s):  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
On Demand ◽  
The World ◽  
Earth Observation Data

scholarly journals Earth Observation for Sustainable Infrastructure: A Review

2021 ◽  
Vol 13 (8) ◽  
pp. 1528
Author(s):  
Yongze Song ◽  
Peng Wu
Keyword(s):  
Sustainable Development ◽  
Large Scale ◽  
Earth Observation ◽  
Future Research ◽  
Observation Data ◽  
Infrastructure Development ◽  
Sustainable Infrastructure ◽  
Development Goals ◽  
Future Research Directions ◽  
Earth Observation Data

Infrastructure is a fundamental sector for sustainable development and Earth observation has great potentials for sustainable infrastructure development (SID). However, implementations of the timely, large–scale and multi–source Earth observation are still limited in satisfying the huge global requirements of SID. This study presents a systematical literature review to identify trends of Earth observation for sustainable infrastructure (EOSI), investigate the relationship between EOSI and Sustainable Development Goals (SDGs), and explore challenges and future directions of EOSI. Results reveal the close associations of infrastructure, urban development, ecosystems, climate, Earth observation and GIS in EOSI, and indicate their relationships. In addition, from the perspective of EOSI–SDGs relationship, the huge potentials of EOSI are demonstrated from the 70% of the infrastructure influenced targets that can be directly or indirectly derived from Earth observation data, but have not been included in current SDG indicators. Finally, typical EOSI cases are presented to indicate challenges and future research directions. This review emphasizes the contributions and potentials of Earth observation to SID and EOSI is a powerful pathway to deliver on SDGs.

scholarly journals Effective, Low-cost Methods of Applying Computer Vision to Public Earth Observation Data

2020 ◽  
Author(s):  
Michael Evans ◽  
Taylor Minich
Keyword(s):  
Wildlife Conservation ◽  
Low Cost ◽  
Ground Truth ◽  
Google Earth ◽  
Earth Observation ◽  
Learning Technologies ◽  
Observation Data ◽  
Solar Arrays ◽  
Observation Systems ◽  
Earth Observation Data

We have an unprecedented ability to analyze and map the Earth&rsquo;s surface, as deep learning technologies are applied to an abundance of Earth observation systems collecting images of the planet daily. In order to realize the potential of these data to improve conservation outcomes, simple, free, and effective methods are needed to enable a wide variety of stakeholders to derive actionable insights from these tools. In this paper we demonstrate simple methods and workflows using free, open computing resources to train well-studied convolutional neural networks and use these to delineate objects of interest in publicly available Earth observation images. With limited training datasets (&lt;1000 observations), we used Google Earth Engine and Tensorflow to process Sentinel-2 and National Agricultural Imaging Program data, and use these to train U-Net and DeepLab models that delineate ground mounted solar arrays and parking lots in satellite imagery. The trained models achieved 81.5% intersection over union between predictions and ground-truth observations in validation images. These images were generated at different times and from different places from those upon which they were trained, indicating the ability of models to generalize outside of data on which they were trained. The two case studies we present illustrate how these methods can be used to inform and improve the development of renewable energy in a manner that is consistent with wildlife conservation.

scholarly journals OPEN FORIS AND GOOGLE EARTH ENGINE LINKING EXPERT PARTICIPATION WITH NATURAL RESOURCE MAPPING AND REMOTE SENSING TRAINING IN TANZANIA

2018 ◽  
Vol XLII-4/W8 ◽  
pp. 117-122 ◽  
Author(s):  
U. Leinonen ◽  
J. Koskinen ◽  
H. Makandi ◽  
E. Mauya ◽  
N. Käyhkö
Keyword(s):  
Remote Sensing ◽  
Open Source ◽  
Open Source Software ◽  
Google Earth ◽  
Earth Observation ◽  
Teaching Staff ◽  
Participatory Mapping ◽  
Observation Data ◽  
Data Repositories ◽  
Google Earth Engine

<p><strong>Abstract.</strong> There is an increasing amount of open Earth observation (EO) data available, offering solutions to map, assess and monitor natural resources and to obtain answers to global and local societal challenges. With the help of free and open source software (FOSS) and open access cloud computing resources, the remote sensing community can take the full advantage of these vast geospatial data repositories. To empower developing societies, support should be given to higher education institutions (HEIs) to train professionals in using the open data, software and tools. In this paper, we describe a participatory mapping methodology, which utilizes open source software Open Foris and QGIS, various open Earth observation data catalogues, and computing capacity of the free Google Earth Engine cloud platform. Using this methodology, we arranged a collaborative data collection event, Mapathon, in Tanzania, followed by a training of the related FOSS tools for HEIs’ teaching staff. We collected feedback from the Mapathon participants about their learning experiences and from teachers about the usability of the methodology in remote sensing training in Tanzania. Based on our experiences and the received feedback, using a participatory mapping campaign as a training method can offer effective learning about environmental remote sensing through a real-world example, as well as networking and knowledge sharing possibilities for the participating group.</p>

scholarly journals GEE TIMESERIES EXPLORER FOR QGIS – INSTANT ACCESS TO PETABYTES OF EARTH OBSERVATION DATA

2021 ◽  
Vol XLVI-4/W2-2021 ◽  
pp. 155-158
Author(s):  
P. Rufin ◽  
A. Rabe ◽  
L. Nill ◽  
P. Hostert
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Satellite Image ◽  
Google Earth ◽  
Earth Observation ◽  
Series Data ◽  
Observation Data ◽  
Local Data ◽  
Small Areas ◽  
Earth Observation Data

Abstract. Earth observation analysis workflows commonly require mass processing of time series data, with data volumes easily exceeding terabyte magnitude, even for relatively small areas of interest. Cloud processing platforms such as Google Earth Engine (GEE) leverage accessibility to satellite image archives and thus facilitate time series analysis workflows. Instant visualization of time series data and integration with local data sources is, however, currently not implemented or requires coding customized scripts or applications. Here, we present the GEE Timeseries Explorer plugin which grants instant access to GEE from within QGIS. It seamlessly integrates the QGIS user interface with a compact widget for visualizing time series from any predefined or customized GEE image collection. Users can visualize time series profiles for a given coordinate as an interactive plot or visualize images with customized band rendering within the QGIS map canvas. The plugin is available through the QGIS plugin repository and detailed documentation is available online (https://geetimeseriesexplorer.readthedocs.io/).

Sign in / Sign up

Export Citation Format

Share Document