Research on Improved Method of Storage and Query of Large-Scale Remote Sensing Images

2018 ◽  
Vol 29 (3) ◽  
pp. 1-16 ◽  
Author(s):  
Jing Weipeng ◽  
Tian Dongxue ◽  
Chen Guangsheng ◽  
Li Yiyuan
Keyword(s):  
Remote Sensing ◽  
Data Storage ◽  
Large Scale ◽  
Remote Sensing Data ◽  
Index Structure ◽  
Hilbert Curve ◽  
Remote Sensing Images ◽  
Storage Model ◽  
Sensing Data ◽  
Low Efficiency

The traditional method is used to deal with massive remote sensing data stored in low efficiency and poor scalability. This article presents a parallel processing method based on MapReduce and HBase. The filling of remote sensing images by the Hilbert curve makes the MapReduce method construct pyramids in parallel to reduce network communication between nodes. Then, the authors design a massive remote sensing data storage model composed of metadata storage model, index structure and filter column family. Finally, this article uses MapReduce frameworks to realize pyramid construction, storage and query of remote sensing data. The experimental results show that this method can effectively improve the speed of data writing and querying, and has good scalability.

scholarly journals RSIMS: Large-Scale Heterogeneous Remote Sensing Images Management System

2021 ◽  
Vol 13 (9) ◽  
pp. 1815
Author(s):  
Xiaohua Zhou ◽  
Xuezhi Wang ◽  
Yuanchun Zhou ◽  
Qinghui Lin ◽  
Jianghua Zhao ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Data Storage ◽  
Management System ◽  
Large Scale ◽  
Distributed Storage ◽  
Storage System ◽  
Remote Sensing Data ◽  
Data Access ◽  
Remote Sensing Images ◽  
Sensing Data

With the remarkable development and progress of earth-observation techniques, remote sensing data keep growing rapidly and their volume has reached exabyte scale. However, it's still a big challenge to manage and process such huge amounts of remote sensing data with complex and diverse structures. This paper designs and realizes a distributed storage system for large-scale remote sensing data storage, access, and retrieval, called RSIMS (remote sensing images management system), which is composed of three sub-modules: RSIAPI, RSIMeta, RSIData. Structured text metadata of different remote sensing images are all stored in RSIMeta based on a set of uniform models, and then indexed by the distributed multi-level Hilbert grids for high spatiotemporal retrieval performance. Unstructured binary image files are stored in RSIData, which provides large scalable storage capacity and efficient GDAL (Geospatial Data Abstraction Library) compatible I/O interfaces. Popular GIS software and tools (e.g., QGIS, ArcGIS, rasterio) can access data stored in RSIData directly. RSIAPI provides users a set of uniform interfaces for data access and retrieval, hiding the complex inner structures of RSIMS. The test results show that RSIMS can store and manage large amounts of remote sensing images from various sources with high and stable performance, and is easy to deploy and use.

scholarly journals RESEARCH ON MULTI-SOURCE SATELLITE IMAGE DATABASE MANAGEMENT SYSTEM

2020 ◽  
Vol XLII-3/W10 ◽  
pp. 565-568
Author(s):  
N. Fu ◽  
L. Sun ◽  
H. Z. Yang ◽  
J. Ma ◽  
B. Q. Liao
Keyword(s):  
Remote Sensing ◽  
Data Storage ◽  
Management System ◽  
Satellite Image ◽  
Remote Sensing Data ◽  
Database Management System ◽  
Sensor Data ◽  
Remote Sensing Images ◽  
Sensing Data ◽  
Effective Organization

Abstract. For the exploration and analysis of electricity, it is necessary to continuously acquire multi-star source, multi-temporal, multi-level remote sensing images for analysis and interpretation. Since the overall data has a variety of features, a data structure for multi-sensor data storage is proposed. On the basis of solving key technologies such as real-time image processing and analysis and remote sensing image normalization processing, the .xml file and remote sensing data geographic information file are used to realize effective organization between remote sensing data and remote sensing data. Based on GDAL design relational database, the formation of a relatively complete management system of data management, shared publishing and application services will maximize the potential value of remote sensing images in electricity remote sensing.

scholarly journals ScienceEarth: A Big Data Platform for Remote Sensing Data Processing

2020 ◽  
Vol 12 (4) ◽  
pp. 607 ◽  
Author(s):  
Chen Xu ◽  
Xiaoping Du ◽  
Zhenzhen Yan ◽  
Xiangtao Fan
Keyword(s):  
Remote Sensing ◽  
Data Processing ◽  
Large Scale ◽  
Cluster Computing ◽  
Remote Sensing Data ◽  
Data Retrieval ◽  
Hilbert Curve ◽  
Sensing Data ◽  
System A ◽  
Efficient Data

Mass remote sensing data management and processing is currently one of the most important topics. In this study, we introduce ScienceEarth, a cluster-based data processing framework. The aim of ScienceEarth is to store, manage, and process large-scale remote sensing data in a cloud-based cluster-computing environment. The platform consists of the following three main parts: ScienceGeoData, ScienceGeoIndex, and ScienceGeoSpark. ScienceGeoData stores and manages remote sensing data. ScienceGeoIndex is an index and query system, a spatial index based on quad-tree and Hilbert curve which is combined for heterogeneous tiled remote sensing data that makes efficient data retrieval in ScienceGeoData. ScienceGeoSpark is an easy-to-use computing framework in which we use Apache Spark as the analytics engine for big remote sensing data processing. The result of tests proves that ScienceEarth can efficiently store, retrieve, and process remote sensing data. The results reveal ScienceEarth has the potential and capabilities of efficient big remote sensing data processing.

scholarly journals An Efficient Decision Support System for Flood Inundation Management Using Intermittent Remote-Sensing Data

2021 ◽  
Vol 13 (14) ◽  
pp. 2818
Author(s):  
Hai Sun ◽  
Xiaoyi Dai ◽  
Wenchi Shou ◽  
Jun Wang ◽  
Xuejing Ruan
Keyword(s):  
Remote Sensing ◽  
Decision Support ◽  
Decision Support System ◽  
Support System ◽  
Water Body ◽  
Remote Sensing Data ◽  
Flood Inundation ◽  
Remote Sensing Images ◽  
Sensing Data ◽  
Tracking Model

Timely acquisition of spatial flood distribution is an essential basis for flood-disaster monitoring and management. Remote-sensing data have been widely used in water-body surveys. However, due to the cloudy weather and complex geomorphic environment, the inability to receive remote-sensing images throughout the day has resulted in some data being missing and unable to provide dynamic and continuous flood inundation process data. To fully and effectively use remote-sensing data, we developed a new decision support system for integrated flood inundation management based on limited and intermittent remote-sensing data. Firstly, we established a new multi-scale water-extraction convolutional neural network named DEU-Net to extract water from remote-sensing images automatically. A specific datasets training method was created for typical region types to separate the water body from the confusing surface features more accurately. Secondly, we built a waterfront contour active tracking model to implicitly describe the flood movement interface. In this way, the flooding process was converted into the numerical solution of the partial differential equation of the boundary function. Space upwind difference format and the time Euler difference format were used to perform the numerical solution. Finally, we established seven indicators that considered regional characteristics and flood-inundation attributes to evaluate flood-disaster losses. The cloud model using the entropy weight method was introduced to account for uncertainties in various parameters. In the end, a decision support system realizing the flood losses risk visualization was developed by using the ArcGIS application programming interface (API). To verify the effectiveness of the model constructed in this paper, we conducted numerical experiments on the model's performance through comparative experiments based on a laboratory scale and actual scale, respectively. The results were as follows: (1) The DEU-Net method had a better capability to accurately extract various water bodies, such as urban water bodies, open-air ponds, plateau lakes etc., than the other comparison methods. (2) The simulation results of the active tracking model had good temporal and spatial consistency with the image extraction results and actual statistical data compared with the synthetic observation data. (3) The application results showed that the system has high computational efficiency and noticeable visualization effects. The research results may provide a scientific basis for the emergency-response decision-making of flood disasters, especially in data-sparse regions.

LESS: LargE-Scale remote sensing data and image simulation framework over heterogeneous 3D scenes

2019 ◽  
Vol 221 ◽  
pp. 695-706 ◽  
Author(s):  
Jianbo Qi ◽  
Donghui Xie ◽  
Tiangang Yin ◽  
Guangjian Yan ◽  
Jean-Philippe Gastellu-Etchegorry ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Large Scale ◽  
Remote Sensing Data ◽  
Image Simulation ◽  
Simulation Framework ◽  
Sensing Data

scholarly journals Quantifying large‐scale ecosystem stability with remote sensing data

2020 ◽  
Vol 6 (3) ◽  
pp. 354-365
Author(s):  
Hannah J. White ◽  
Willson Gaul ◽  
Dinara Sadykova ◽  
Lupe León‐Sánchez ◽  
Paul Caplat ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Large Scale ◽  
Remote Sensing Data ◽  
Ecosystem Stability ◽  
Sensing Data

Time-series processing of large scale remote sensing data with extreme learning machine

2014 ◽  
Vol 128 ◽  
pp. 199-206 ◽  
Author(s):  
Jiaoyan Chen ◽  
Guozhou Zheng ◽  
Cong Fang ◽  
Ningyu Zhang ◽  
Huajun Chen ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Time Series ◽  
Extreme Learning Machine ◽  
Large Scale ◽  
Remote Sensing Data ◽  
Sensing Data ◽  
Learning Machine
Sign in / Sign up

Export Citation Format

Share Document