Determinants of large-scale spatial data processing in Polish mining

Geological modeling methods are widely used to represent subsurface structures for a multitude of applications &#8211; from scientific investigations, over natural resource and reservoir studies, to large-scale analyses and geological representations by geological surveys. In recent years, we have seen an increase in the availability of geological modeling methods. However, many of these methods are difficult to use due to preliminary data processing steps, which can be specifically difficult for geoscientific data in geographic coordinate systems.We attempt to simplify the access to open-source spatial data processing for geological modeling with the development of GemGIS, a Python-based open-source library. GemGIS wraps and extends the functionality of packages known to the geo-community such as GeoPandas, Rasterio, OWSLib, Shapely, PyVista, Pandas, NumPy and the geomodelling package GemPy. The aim of GemGIS, as indicated by the name, is to become a bridge between conventional geoinformation systems (GIS) such as ArcGIS and QGIS, and geomodelling tools such as GemPy, allowing simpler and more automated workflows from one environment to the other.Data within the different disciplines of geosciences are often available in a variety of data formats that need to be converted or transformed for visualization in 2D and 3D and subsequent geomodelling methods. This is where GemGIS comes into play. GemGIS is capable of working with vector data created in GIS systems through GeoPandas, Pandas and Shapely, with raster data through rasterio and NumPy, with data obtained from web services such as maps or digital elevation models through OWSLib and with meshes through PyVista. Support for geophysical data and additional geo-formats are constantly added.The GemGIS package already contains several tutorials explaining how the different modules can be used to process spatial data. It was decided against creating new data classes in case users are already familiar with concepts such as (Geo-)DataFrames in (Geo-)Pandas or PolyData/Grids in PyVista.The GemGIS package is hosted at https://github.com/cgre-aachen/gemgis, the documentation is available at https://gemgis.readthedocs.io/en/latest/index.html. GemGIS is also available on PyPi. You can install GemGIS in your Python environment using &#8216;pip install gemgis&#8217;.We welcome contributions to the project through pull requests and are open to suggestions and comments, also over Github issues, especially about possible links to other existing software developments and approaches to integrate geoscientific data processing and geomodelling.

Download Full-text

Infrastructural approach to processing of spatial data in the problems of management of territorial development

Вычислительные технологии ◽

10.25743/ict.2018.23.16488 ◽

2018 ◽

Author(s):

И.В. Бычков ◽

Г.М. Ружников ◽

В.В. Парамонов ◽

А.С. Шумилов ◽

Р.К. Фёдоров

Keyword(s):

Data Processing ◽

Spatial Data ◽

Data Exchange ◽

Complex Analysis ◽

Data Entry ◽

Content Management ◽

Volume Data ◽

Territorial Development ◽

Distributed Services ◽

Spatial Data Processing

Рассмотрен инфраструктурный подход обработки пространственных данных для решения задач управления территориальным развитием, который основан на сервис-ориентированной парадигме, стандартах OGC, web-технологиях, WPS-сервисах и геопортале. The development of territories is a multi-dimensional and multi-aspect process, which can be characterized by large volumes of financial, natural resources, social, ecological and economic data. The data is highly localized and non-coordinated, which limits its complex analysis and usage. One of the methods of large volume data processing is information-analytical environments. The architecture and implementation of the information-analytical environment of the territorial development in the form of Geoportal is presented. Geoportal provides software instruments for spatial and thematic data exchange for its users, as well as OGC-based distributed services that deal with the data processing. Implementation of the processing and storing of the data in the form of services located on distributed servers allows simplifying their updating and maintenance. In addition, it allows publishing and makes processing to be more open and controlled process. Geoportal consists of following modules: content management system Calipso (presentation of user interface, user management, data visualization), RDBMS PostgreSQL with spatial data processing extension, services of relational data entry and editing, subsystem of launching and execution of WPS-services, as well as services of spatial data processing, deployed at the local cloud environment. The presented article states the necessity of using the infrastructural approach when creating the information-analytical environment for the territory management, which is characterized by large volumes of spatial and thematical data that needs to be processed. The data is stored in various formats and applications of service-oriented paradigm, OGC standards, web-technologies, Geoportal and distributed WPS-services. The developed software system was tested on a number of tasks that arise during the territory development.

Download Full-text

UAV and terrestrial laser scanner data processing for large scale topographic mapping

Mongolian Geoscientist ◽

10.5564/mgs.v50i0.1329 ◽

2020 ◽

Vol 50 ◽

pp. 63-73

Author(s):

Ganbold Ulziisaikhan ◽

Dash Oyuntsetseg

Keyword(s):

Data Processing ◽

Spatial Data ◽

Large Scale ◽

Laser Scanner ◽

Measurement Technology ◽

Topographic Mapping ◽

Terrestrial Laser Scanner ◽

Scanner Data ◽

Processing Scheme ◽

Digital Elevation

Integrating spatial data from different sources results in visualization, which is the last step in the process of digital basic topographic map creation. Digital elevation model and visualization will used for geomorphological mapping, geospatial database, urban planning and etc. Large scale topographic mapping in the world countries is really a prominent challenge in geospatial industries today. The purpose of this work is to integrate laser scanner data with the ones generated by aerial photogrammetry from UAV, to produce detailed maps that can used by geodetic engineers to optimize their analysis. In addition, terrestrial - based LiDAR scans and UAV photogrammetric data were collected in Sharga hill in the north zone of Mongolia. In this paper, different measurement technology and processing software systems combined for topographic mapping in the data processing scheme. UTM (Universal Transverse Mercator) projected coordinate system calculated in WGS84 reference ellipsoid. Feature compilation involving terrestrial laser scanner data and UAV data will integrated to offer Digital Elevation Models (DEM) as the main interest of the topographic mapping activity. Used UAV generate high-resolution orthomosaics and detailed 3D models of areas where no data, are available. That result issued to create topographic maps with a scale of 1:1000 of geodetic measurements. Preliminary results indicate that discontinuity data collection from UAV closely matches the data collected using laser scanner.

Download Full-text

DEVELOPMENT OF A HETEROGENIC DISTRIBUTED ENVIRONMENT FOR SPATIAL DATA PROCESSING USING CLOUD TECHNOLOGIES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b4-385-2016 ◽

2016 ◽

Vol XLI-B4 ◽

pp. 385-390

Author(s):

A. S. Garov ◽

I. P. Karachevtseva ◽

E. V. Matveev ◽

A. E. Zubarev ◽

I. V. Florinsky

Keyword(s):

Data Processing ◽

Spatial Data ◽

3D Visualization ◽

New Technology ◽

Main Idea ◽

Remote Sensing Data ◽

Data Access ◽

Working Environment ◽

Software Environment ◽

Spatial Data Processing

We are developing a unified distributed communication environment for processing of spatial data which integrates web-, desktop- and mobile platforms and combines volunteer computing model and public cloud possibilities. The main idea is to create a flexible working environment for research groups, which may be scaled according to required data volume and computing power, while keeping infrastructure costs at minimum. It is based upon the "single window" principle, which combines data access via geoportal functionality, processing possibilities and communication between researchers. Using an innovative software environment the recently developed planetary information system (<a href="http://cartsrv.mexlab.ru/geoportal"target="_blank">http://cartsrv.mexlab.ru/geoportal</a>) will be updated. The new system will provide spatial data processing, analysis and 3D-visualization and will be tested based on freely available Earth remote sensing data as well as Solar system planetary images from various missions. Based on this approach it will be possible to organize the research and representation of results on a new technology level, which provides more possibilities for immediate and direct reuse of research materials, including data, algorithms, methodology, and components. The new software environment is targeted at remote scientific teams, and will provide access to existing spatial distributed information for which we suggest implementation of a user interface as an advanced front-end, e.g., for virtual globe system.

Download Full-text