scholarly journals An Overview of Platforms for Big Earth Observation Data Management and Analysis

2020 ◽  
Vol 12 (8) ◽  
pp. 1253 ◽  
Author(s):  
Vitor Gomes ◽  
Gilberto Queiroz ◽  
Karine Ferreira
Keyword(s):  
Data Management ◽  
Data Access ◽  
Database Systems ◽  
Google Earth ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Novel Technologies ◽  
Community Interest ◽  
Earth Observation Data

In recent years, Earth observation (EO) satellites have generated big amounts of geospatial data that are freely available for society and researchers. This scenario brings challenges for traditional spatial data infrastructures (SDI) to properly store, process, disseminate and analyze these big data sets. To meet these demands, novel technologies have been proposed and developed, based on cloud computing and distributed systems, such as array database systems, MapReduce systems and web services to access and process big Earth observation data. Currently, these technologies have been integrated into cutting edge platforms in order to support a new generation of SDI for big Earth observation data. This paper presents an overview of seven platforms for big Earth observation data management and analysis—Google Earth Engine (GEE), Sentinel Hub, Open Data Cube (ODC), System for Earth Observation Data Access, Processing and Analysis for Land Monitoring (SEPAL), openEO, JEODPP, and pipsCloud. We also provide a comparison of these platforms according to criteria that represent capabilities of the EO community interest.

Analyzing large-scale Earth Observation data repositories made simple with OpenEO Platform

2021 ◽  
Author(s):  
Edzer Pebesma ◽  
Patrick Griffiths ◽  
Christian Briese ◽  
Alexander Jacob ◽  
Anze Skerlevaj ◽  
...  
Keyword(s):  
Large Scale ◽  
Virtual Machines ◽  
Google Earth ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Repositories ◽  
Software Ecosystem ◽  
Wide Range ◽  
Earth Observation Data

<p>The OpenEO API allows the analysis of large amounts of Earth Observation data using a high-level abstraction of data and processes. Rather than focusing on the management of virtual machines and millions of imagery files, it allows to create jobs that take a spatio-temporal section of an image collection (such as Sentinel L2A), and treat it as a data cube. Processes iterate or aggregate over pixels, spatial areas, spectral bands, or time series, while working at arbitrary spatial resolution. This pattern, pioneered by Google Earth Engine™ (GEE), lets the user focus on the science rather than on data management.</p><p>The openEO H2020 project (2017-2020) has developed the API as well as an ecosystem of software around it, including clients (JavaScript, Python, R, QGIS, browser-based), back-ends that translate API calls into existing image analysis or GIS software or services (for Sentinel Hub, WCPS, Open Data Cube, GRASS GIS, GeoTrellis/GeoPySpark, and GEE) as well as a hub that allows querying and searching openEO providers for their capabilities and datasets. The project demonstrated this software in a number of use cases, where identical processing instructions were sent to different implementations, allowing comparison of returned results.</p><p>A follow-up, ESA-funded project “openEO Platform” realizes the API and progresses the software ecosystem into operational services and applications that are accessible to everyone, that involve federated deployment (using the clouds managed by EODC, Terrascope, CreoDIAS and EuroDataCube), that will provide payment models (“pay per compute job”) conceived and implemented following the user community needs and that will use the EOSC (European Open Science Cloud) marketplace for dissemination and authentication. A wide range of large-scale cases studies will demonstrate the ability of the openEO Platform to scale to large data volumes.  The case studies to be addressed include on-demand ARD generation for SAR and multi-spectral data, agricultural demonstrators like crop type and condition monitoring, forestry services like near real time forest damage assessment as well as canopy cover mapping, environmental hazard monitoring of floods and air pollution as well as security applications in terms of vessel detection in the mediterranean sea.</p><p>While the landscape of cloud-based EO platforms and services has matured and diversified over the past decade, we believe there are strong advantages for scientists and government agencies to adopt the openEO approach. Beyond the absence of vendor/platform lock-in or EULA’s we mention the abilities to (i) run arbitrary user code (e.g. written in R or Python) close to the data, (ii) carry out scientific computations on an entirely open source software stack, (iii) integrate different platforms (e.g., different cloud providers offering different datasets), and (iv) help create and extend this software ecosystem. openEO uses the OpenAPI standard, aligns with modern OGC API standards, and uses the STAC (SpatioTemporal Asset Catalog) to describe image collections and image tiles.</p>

scholarly journals Achieving the Full Vision of Earth Observation Data Cubes

Data ◽  
2019 ◽  
Vol 4 (3) ◽  
pp. 94 ◽  
Author(s):  
Steve Kopp ◽  
Peter Becker ◽  
Abhijit Doshi ◽  
Dawn J. Wright ◽  
Kaixi Zhang ◽  
...  
Keyword(s):  
Image Data ◽  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
Data Cubes ◽  
Multiple Data ◽  
Storage Location ◽  
Science Community ◽  
Earth Observation Data ◽  
And Storage

Earth observation imagery have traditionally been expensive, difficult to find and access, and required specialized skills and software to transform imagery into actionable information. This has limited adoption by the broader science community. Changes in cost of imagery and changes in computing technology over the last decade have enabled a new approach for how to organize, analyze, and share Earth observation imagery, broadly referred to as a data cube. The vision and promise of image data cubes is to lower these hurdles and expand the user community by making analysis ready data readily accessible and providing modern approaches to more easily analyze and visualize the data, empowering a larger community of users to improve their knowledge of place and make better informed decisions. Image data cubes are large collections of temporal, multivariate datasets typically consisting of analysis ready multispectral Earth observation data. Several flavors and variations of data cubes have emerged. To simplify access for end users we developed a flexible approach supporting multiple data cube styles, referencing images in their existing structure and storage location, enabling fast access, visualization, and analysis from a wide variety of web and desktop applications. We provide here an overview of that approach and three case studies.

scholarly journals Data Cube on Demand (DCoD): Generating an earth observation Data Cube anywhere in the world

2020 ◽  
Vol 87 ◽  
pp. 102035 ◽  
Author(s):  
Gregory Giuliani ◽  
Bruno Chatenoux ◽  
Thomas Piller ◽  
Frédéric Moser ◽  
Pierre Lacroix
Keyword(s):  
Earth Observation ◽  
Data Cube ◽  
Observation Data ◽  
On Demand ◽  
The World ◽  
Earth Observation Data

scholarly journals Big Earth data: disruptive changes in Earth observation data management and analysis?

2019 ◽  
Vol 13 (7) ◽  
pp. 832-850 ◽  
Author(s):  
Martin Sudmanns ◽  
Dirk Tiede ◽  
Stefan Lang ◽  
Helena Bergstedt ◽  
Georg Trost ◽  
...  
Keyword(s):  
Data Management ◽  
Earth Observation ◽  
Observation Data ◽  
Earth Observation Data

MAAP: The Mission Algorithm and Analysis Platform: A New Virtual and Collaborative Environment for the Scientific Community

2020 ◽  
Author(s):  
Clement Albinet ◽  
Sebastien Nouvellon ◽  
Björn Frommknecht ◽  
Roger Rutakaza ◽  
Sandrine Daniel ◽  
...  
Keyword(s):  
Scientific Community ◽  
Forest Biomass ◽  
Data Access ◽  
Earth Observation ◽  
Observation Data ◽  
Societal Benefit ◽  
Collaborative Environment ◽  
Earth Observation Data ◽  
Earth Observation Satellites ◽  
Analysis Platform

<p>The ESA-NASA multi-Mission Algorithm and Analysis Platform (MAAP) is dedicated to the BIOMASS [1], NISAR [2] and GEDI [3] missions. This analysis platform will be a virtual open and collaborative environment. The main goal is to bring together data centres (Earth Observation and non-Earth Observation data), computing resources and hosted processing in order to better address the needs of scientists and federate the scientific community.</p><p>The MAAP will provide functions to access data and metadata from different sources such as Earth observation satellites data from science missions; visualisation functions to display the results of the system processing (trends, graphs, maps ...) and results of statistic and analysis tools; collaborative functions to share data, algorithms, ideas between the MAAP users; processing functions including development environments and an orchestration system allowing to create and run processing chains from official algorithms.</p><p>Currently, the MAAP is in its pilot phase. The architecture for the MAAP pilot foresees two independent elements, one developed by ESA, one developed by NASA, unified by a common user entry point. Both elements will be deployed on Cloud infrastructures. Interoperability between the elements is envisaged for data discovery, data access and identity and access management.</p><p>The ESA element architecture is based on technical solutions including: Microservices, Docker images, Kubernetes; Cloud-based virtual development environments (such as Jupyter or Eclipse CHE) for the MAAP algorithm developers; a framework to create, run and monitor chains of algorithms containerised as docker images. Interoperability between both ESA and NASA elements will be based on CMR (NASA Common Metadata Repository), services bases on OGC standards (such as WMS/WMTS, WCS and WPS) and secured with the OAUTH2 protocol.</p><p>This presentation focuses on the pilot platform and how interoperability between the NASA and ESA elements will be achieved. It also gives insight into the architecture of the ESA element and the technical implementation of this virtual environment. Finally, it will present the very first achievements and return of experience of the pilot platform.</p><p> </p><p><strong>REFERENCES</strong></p><p>[1] T. Le Toan, S. Quegan, M. Davidson, H. Balzter, P. Paillou, K. Papathanassiou, S. Plummer, F. Rocca, S. Saatchi, H. Shugart and L. Ulander, “The BIOMASS Mission: Mapping global forest biomass to better understand the terrestrial carbon cycle”, Remote Sensing of Environment, Vol. 115, No. 11, pp. 2850-2860, June 2011.</p><p>[2] P.A. Rosen, S. Hensley, S. Shaffer, L. Veilleux, M. Chakraborty, T. Misra, R. Bhan, V. Raju Sagi and R. Satish, "The NASA-ISRO SAR mission - An international space partnership for science and societal benefit", IEEE Radar Conference (RadarCon), pp. 1610-1613, 10-15 May 2015.</p><p>[3] https://science.nasa.gov/missions/gedi</p>

scholarly journals Effective, Low-cost Methods of Applying Computer Vision to Public Earth Observation Data

2020 ◽  
Author(s):  
Michael Evans ◽  
Taylor Minich
Keyword(s):  
Wildlife Conservation ◽  
Low Cost ◽  
Ground Truth ◽  
Google Earth ◽  
Earth Observation ◽  
Learning Technologies ◽  
Observation Data ◽  
Solar Arrays ◽  
Observation Systems ◽  
Earth Observation Data

We have an unprecedented ability to analyze and map the Earth’s surface, as deep learning technologies are applied to an abundance of Earth observation systems collecting images of the planet daily. In order to realize the potential of these data to improve conservation outcomes, simple, free, and effective methods are needed to enable a wide variety of stakeholders to derive actionable insights from these tools. In this paper we demonstrate simple methods and workflows using free, open computing resources to train well-studied convolutional neural networks and use these to delineate objects of interest in publicly available Earth observation images. With limited training datasets (<1000 observations), we used Google Earth Engine and Tensorflow to process Sentinel-2 and National Agricultural Imaging Program data, and use these to train U-Net and DeepLab models that delineate ground mounted solar arrays and parking lots in satellite imagery. The trained models achieved 81.5% intersection over union between predictions and ground-truth observations in validation images. These images were generated at different times and from different places from those upon which they were trained, indicating the ability of models to generalize outside of data on which they were trained. The two case studies we present illustrate how these methods can be used to inform and improve the development of renewable energy in a manner that is consistent with wildlife conservation.

scholarly journals GEE TIMESERIES EXPLORER FOR QGIS – INSTANT ACCESS TO PETABYTES OF EARTH OBSERVATION DATA

2021 ◽  
Vol XLVI-4/W2-2021 ◽  
pp. 155-158
Author(s):  
P. Rufin ◽  
A. Rabe ◽  
L. Nill ◽  
P. Hostert
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Satellite Image ◽  
Google Earth ◽  
Earth Observation ◽  
Series Data ◽  
Observation Data ◽  
Local Data ◽  
Small Areas ◽  
Earth Observation Data

Abstract. Earth observation analysis workflows commonly require mass processing of time series data, with data volumes easily exceeding terabyte magnitude, even for relatively small areas of interest. Cloud processing platforms such as Google Earth Engine (GEE) leverage accessibility to satellite image archives and thus facilitate time series analysis workflows. Instant visualization of time series data and integration with local data sources is, however, currently not implemented or requires coding customized scripts or applications. Here, we present the GEE Timeseries Explorer plugin which grants instant access to GEE from within QGIS. It seamlessly integrates the QGIS user interface with a compact widget for visualizing time series from any predefined or customized GEE image collection. Users can visualize time series profiles for a given coordinate as an interactive plot or visualize images with customized band rendering within the QGIS map canvas. The plugin is available through the QGIS plugin repository and detailed documentation is available online (https://geetimeseriesexplorer.readthedocs.io/).

Sign in / Sign up

Export Citation Format

Share Document