China Data Cube (CDC) for Big Earth Observation Data: Lessons Learned from the Design and Implementation

2018 ◽  
Author(s):  
Xiaochuang Yao ◽  
Yingbo Liu ◽  
Qianqian Cao ◽  
Junjie Li ◽  
Ruihong Huang ◽  
...  
Keyword(s):  
Earth Observation ◽  
Lessons Learned ◽  
Data Cube ◽  
Observation Data ◽  
Design And Implementation ◽  
Earth Observation Data

Analyzing large-scale Earth Observation data repositories made simple with OpenEO Platform

2021 ◽  
Author(s):  
Edzer Pebesma ◽  
Patrick Griffiths ◽  
Christian Briese ◽  
Alexander Jacob ◽  
Anze Skerlevaj ◽  
...  
Keyword(s):  
Large Scale ◽  
Virtual Machines ◽  
Google Earth ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Repositories ◽  
Software Ecosystem ◽  
Wide Range ◽  
Earth Observation Data

<p>The OpenEO API allows the analysis of large amounts of Earth Observation data using a high-level abstraction of data and processes. Rather than focusing on the management of virtual machines and millions of imagery files, it allows to create jobs that take a spatio-temporal section of an image collection (such as Sentinel L2A), and treat it as a data cube. Processes iterate or aggregate over pixels, spatial areas, spectral bands, or time series, while working at arbitrary spatial resolution. This pattern, pioneered by Google Earth Engine™ (GEE), lets the user focus on the science rather than on data management.</p><p>The openEO H2020 project (2017-2020) has developed the API as well as an ecosystem of software around it, including clients (JavaScript, Python, R, QGIS, browser-based), back-ends that translate API calls into existing image analysis or GIS software or services (for Sentinel Hub, WCPS, Open Data Cube, GRASS GIS, GeoTrellis/GeoPySpark, and GEE) as well as a hub that allows querying and searching openEO providers for their capabilities and datasets. The project demonstrated this software in a number of use cases, where identical processing instructions were sent to different implementations, allowing comparison of returned results.</p><p>A follow-up, ESA-funded project “openEO Platform” realizes the API and progresses the software ecosystem into operational services and applications that are accessible to everyone, that involve federated deployment (using the clouds managed by EODC, Terrascope, CreoDIAS and EuroDataCube), that will provide payment models (“pay per compute job”) conceived and implemented following the user community needs and that will use the EOSC (European Open Science Cloud) marketplace for dissemination and authentication. A wide range of large-scale cases studies will demonstrate the ability of the openEO Platform to scale to large data volumes.  The case studies to be addressed include on-demand ARD generation for SAR and multi-spectral data, agricultural demonstrators like crop type and condition monitoring, forestry services like near real time forest damage assessment as well as canopy cover mapping, environmental hazard monitoring of floods and air pollution as well as security applications in terms of vessel detection in the mediterranean sea.</p><p>While the landscape of cloud-based EO platforms and services has matured and diversified over the past decade, we believe there are strong advantages for scientists and government agencies to adopt the openEO approach. Beyond the absence of vendor/platform lock-in or EULA’s we mention the abilities to (i) run arbitrary user code (e.g. written in R or Python) close to the data, (ii) carry out scientific computations on an entirely open source software stack, (iii) integrate different platforms (e.g., different cloud providers offering different datasets), and (iv) help create and extend this software ecosystem. openEO uses the OpenAPI standard, aligns with modern OGC API standards, and uses the STAC (SpatioTemporal Asset Catalog) to describe image collections and image tiles.</p>

scholarly journals Achieving the Full Vision of Earth Observation Data Cubes

Data ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. 94 ◽  
Author(s):  
Steve Kopp ◽  
Peter Becker ◽  
Abhijit Doshi ◽  
Dawn J. Wright ◽  
Kaixi Zhang ◽  
...  
Keyword(s):  
Image Data ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Cubes ◽  
Multiple Data ◽  
Storage Location ◽  
Science Community ◽  
Earth Observation Data ◽  
And Storage

Earth observation imagery have traditionally been expensive, difficult to find and access, and required specialized skills and software to transform imagery into actionable information. This has limited adoption by the broader science community. Changes in cost of imagery and changes in computing technology over the last decade have enabled a new approach for how to organize, analyze, and share Earth observation imagery, broadly referred to as a data cube. The vision and promise of image data cubes is to lower these hurdles and expand the user community by making analysis ready data readily accessible and providing modern approaches to more easily analyze and visualize the data, empowering a larger community of users to improve their knowledge of place and make better informed decisions. Image data cubes are large collections of temporal, multivariate datasets typically consisting of analysis ready multispectral Earth observation data. Several flavors and variations of data cubes have emerged. To simplify access for end users we developed a flexible approach supporting multiple data cube styles, referencing images in their existing structure and storage location, enabling fast access, visualization, and analysis from a wide variety of web and desktop applications. We provide here an overview of that approach and three case studies.

scholarly journals Data Cube on Demand (DCoD): Generating an earth observation Data Cube anywhere in the world

2020 ◽  
Vol 87 ◽  
pp. 102035 ◽  
Author(s):  
Gregory Giuliani ◽  
Bruno Chatenoux ◽  
Thomas Piller ◽  
Frédéric Moser ◽  
Pierre Lacroix
Keyword(s):  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
On Demand ◽  
The World ◽  
Earth Observation Data

The Sen2Cube.at national semantic Earth observation data cube for Austria 

2021 ◽  
Author(s):  
Martin Sudmanns ◽  
Hannah Augustin ◽  
Lucas van der Meer ◽  
Andrea Baraldi ◽  
Dirk Tiede
Keyword(s):  
Inference Engine ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Semantic Enrichment ◽  
Area Of Interest ◽  
Information Layer ◽  
Semantic Models ◽  
Earth Observation Data ◽  
Sentinel 2

<div> <p>The Sen2Cube.at is a Sentinel-2 semantic Earth observation (EO) data and information cube that combines an EO data cube with an AI-based inference engine by integrating a computer-vision approach to infer new information. Our approach uses semantic enrichment of optical images and makes the data and information directly available and accessible for further use within an EO data cube. The architecture is based on an expert system, in which domain-knowledge can be encoded in semantic models (knowledgebase) and applied to the Sentinel-2 data as well as semantically enriched, data-derived information (factbase).  </p> </div><div> <p>The initial semantic enrichment in the Sen2Cube.at system is general-purpose, user- and application-independent, derived directly from optical EO images as an initial step towards a scene classification map. These information layers are automatically generated from Sentinel-2 images with the SIAM software (Satellite Image Automated Mapper). SIAM is a knowledge-based and physical-model-based decision tree that produces a set of information layers in a fully automated process that is applicable worldwide and does not require any samples. A graphical inference engine allows application-specific Web-based semantic querying based on the generic information layer as a replicable and explainable approach to produce information. The graphical inference engine is a new Browser-based graphical user interface (GUI) developed in-house with a semantic querying language. Users formulate semantic models in a graphical way and can execute them on any area-of-interest and time interval, which will be evaluated by the core of the inference engine attached to the data cube. This also enables non-expert users to formulate analyses without requiring programming skills.  </p> </div><div> <p>While the methodology is software-independent, the prototype is based on the Open Data Cube and additional in-house developed components in the Python programming language. Scaling is possible depending on the available infrastructure resources due to the system’s Docker-based container architecture. Through its fully automated semantic enrichment, innovative graphical querying language in the GUI for semantic querying and analysis as well as the implementation as a scalable infrastructure, this approach is suited for big data analysis of Earth observation data. It was successfully scaled to a national data cube for Austria, containing all available Sentinel-2 images from the platforms A and B. </p> </div>

scholarly journals Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes

Data ◽  
2019 ◽  
Vol 4 (4) ◽  
pp. 147 ◽  
Author(s):  
Gregory Giuliani ◽  
Gilberto Camara ◽  
Brian Killough ◽  
Stuart Minchin
Keyword(s):  
Open Science ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Special Issue ◽  
Data Repositories ◽  
Data Cubes ◽  
Information Power ◽  
Wide Range ◽  
Earth Observation Data

Earth Observation Data Cubes (EODC) have emerged as a promising solution to efficiently and effectively handle Big Earth Observation (EO) Data generated by satellites and made freely and openly available from different data repositories. The aim of this Special Issue, “Earth Observation Data Cube”, in Data, is to present the latest advances in EODC development and implementation, including innovative approaches for the exploitation of satellite EO data using multi-dimensional (e.g., spatial, temporal, spectral) approaches. This Special Issue contains 14 articles covering a wide range of topics such as Synthetic Aperture Radar (SAR), Analysis Ready Data (ARD), interoperability, thematic applications (e.g., land cover, snow cover mapping), capacity development, semantics, processing techniques, as well as national implementations and best practices. These papers made significant contributions to the advancement of a more Open and Reproducible Earth Observation Science, reducing the gap between users’ expectations for decision-ready products and current Big Data analytical capabilities, and ultimately unlocking the information power of EO data by transforming them into actionable knowledge.

scholarly journals Essential Variables for Environmental Monitoring: What Are the Possible Contributions of Earth Observation Data Cubes?

Data ◽  
2020 ◽  
Vol 5 (4) ◽  
pp. 100 ◽  
Author(s):  
Gregory Giuliani ◽  
Elvire Egger ◽  
Julie Italiano ◽  
Charlotte Poussin ◽  
Jean-Philippe Richard ◽  
...  
Keyword(s):  
Environmental Monitoring ◽  
Satellite Data ◽  
Environmental Sustainability ◽  
Environmental Changes ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Temporal Dimension ◽  
Global Issue ◽  
Earth Observation Data

Environmental sustainability is nowadays a major global issue that requires efficient and effective responses from governments. Essential variables (EV) have emerged in different scientific communities as a means to characterize and follow environmental changes through a set of measurements required to support policy evidence. To help track these changes, our planet has been under continuous observation from satellites since 1972. Currently, petabytes of satellite Earth observation (EO) data are freely available. However, the full information potential of EO data has not been yet realized because many big data challenges and complexity barriers hinder their effective use. Consequently, facilitating the production of EVs using the wealth of satellite EO data can be beneficial for environmental monitoring systems. In response to this issue, a comprehensive list of EVs that can take advantage of consistent time-series satellite data has been derived. In addition, a set of use-cases, using an Earth Observation Data Cube (EODC) to process large volumes of satellite data, have been implemented to demonstrate the practical applicability of EODC to produce EVs. The proposed approach has been successfully tested showing that EODC can facilitate the production of EVs at different scales and benefiting from the spatial and temporal dimension of satellite EO data for enhanced environmental monitoring.

scholarly journals Semantic Earth Observation Data Cubes

Data ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. 102 ◽  
Author(s):  
Hannah Augustin ◽  
Martin Sudmanns ◽  
Dirk Tiede ◽  
Stefan Lang ◽  
Andrea Baraldi
Keyword(s):  
Data Retrieval ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Cubes ◽  
Sensory Data ◽  
Main Challenge ◽  
Data Content ◽  
Spatio Temporal ◽  
Earth Observation Data

There is an increasing amount of free and open Earth observation (EO) data, yet more information is not necessarily being generated from them at the same rate despite high information potential. The main challenge in the big EO analysis domain is producing information from EO data, because numerical, sensory data have no semantic meaning; they lack semantics. We are introducing the concept of a semantic EO data cube as an advancement of state-of-the-art EO data cubes. We define a semantic EO data cube as a spatio-temporal data cube containing EO data, where for each observation at least one nominal (i.e., categorical) interpretation is available and can be queried in the same instance. Here we clarify and share our definition of semantic EO data cubes, demonstrating how they enable different possibilities for data retrieval, semantic queries based on EO data content and semantically enabled analysis. Semantic EO data cubes are the foundation for EO data expert systems, where new information can be inferred automatically in a machine-based way using semantic queries that humans understand. We argue that semantic EO data cubes are better positioned to handle current and upcoming big EO data challenges than non-semantic EO data cubes, while facilitating an ever-diversifying user-base to produce their own information and harness the immense potential of big EO data.

Usage of Earth Observation for Solar Energy Market Development: Lessons Learned

Solar Energy ◽  
2006 ◽  
Author(s):  
M. Schroedter-Homscheidt ◽  
S. Bofinger ◽  
H. Breitkreuz ◽  
G. Heilscher ◽  
S. Stettler
Keyword(s):  
Solar Energy ◽  
Investment Decision ◽  
European Space Agency ◽  
Earth Observation ◽  
Lessons Learned ◽  
Market Development ◽  
Plant Management ◽  
Observation Data ◽  
Large Set ◽  
Earth Observation Data

Within the Earth Observation Market Development (EOMD) program of the European Space Agency (ESA) the ENVISOLAR project aims at an intensified usage of earth observation based information products in the solar energy industries. Existing services for investment decision, plant management, load forecasting, and science and consulting rely on high quality surface solar irradiance measurements and reliable processing chains to deliver such information regularly. Requirements for earth observation data as well as blockages preventing their use have been identified. In consequence, existing data processing chains are analyzed as to their conformity with the needs of the solar industry. The paper focuses on how earth observation itself can contribute to the market development of solar energy technologies. Issues like quality of irradiance data for planning and managing solar energy systems, reliability and availability of earth observation information, requirements as to temporal, spatial and spectral resolution of earth observation data, and the cost-effectiveness of satellite based information compared to maintenance costs for a large set of on-site measurement devices are addressed.

scholarly journals An Overview of Platforms for Big Earth Observation Data Management and Analysis

2020 ◽  
Vol 12 (8) ◽  
pp. 1253 ◽  
Author(s):  
Vitor Gomes ◽  
Gilberto Queiroz ◽  
Karine Ferreira
Keyword(s):  
Data Management ◽  
Data Access ◽  
Database Systems ◽  
Google Earth ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Novel Technologies ◽  
Community Interest ◽  
Earth Observation Data

In recent years, Earth observation (EO) satellites have generated big amounts of geospatial data that are freely available for society and researchers. This scenario brings challenges for traditional spatial data infrastructures (SDI) to properly store, process, disseminate and analyze these big data sets. To meet these demands, novel technologies have been proposed and developed, based on cloud computing and distributed systems, such as array database systems, MapReduce systems and web services to access and process big Earth observation data. Currently, these technologies have been integrated into cutting edge platforms in order to support a new generation of SDI for big Earth observation data. This paper presents an overview of seven platforms for big Earth observation data management and analysis—Google Earth Engine (GEE), Sentinel Hub, Open Data Cube (ODC), System for Earth Observation Data Access, Processing and Analysis for Land Monitoring (SEPAL), openEO, JEODPP, and pipsCloud. We also provide a comparison of these platforms according to criteria that represent capabilities of the EO community interest.

Sign in / Sign up

Export Citation Format

Share Document