Earthquake Data Archiving and Retrieval System; archived data sets in the standardized library, SL000001 to SL000100

Abstract With the rapid growth of the volume of video data and the development of multimedia technologies, it has become necessary to have the ability to accurately and quickly browse and search through information stored in large multimedia databases. For this purpose, content-based video retrieval ( CBVR ) has become an active area of research over the last decade. In this paper, We propose a content-based video retrieval system providing similar videos from a large multimedia data-set based on a query video. The approach uses vector motion-based signatures to describe the visual content and uses machine learning techniques to extract key-frames for rapid browsing and efficient video indexing. We have implemented the proposed approach on both, single machine and real-time distributed cluster to evaluate the real-time performance aspect, especially when the number and size of videos are large. Experiments are performed using various benchmark action and activity recognition data-sets and the results reveal the effectiveness of the proposed method in both accuracy and processing time compared to state-of-the-art methods.

Download Full-text

Biogeochemical climatology for the Southern Benguela Upwelling System, constructed from <i>in situ</i> monitoring data

10.5194/essd-2018-60 ◽

2018 ◽

Author(s):

Stephanie de Villiers

Keyword(s):

Monitoring Data ◽

Data Sets ◽

Profile Data ◽

Data Archiving ◽

Upwelling System ◽

In Situ Data ◽

Benguela Upwelling ◽

Phosphate Nitrate ◽

Insight Into

Abstract. An annual and a seasonal biogeochemical climatology had been constructed for the Southern Benguela Upwelling System, from in situ data collected along a 12 station monitoring line, sampled at monthly intervals from 2001 to 2012. The monitoring line reaches a maximum offshore distance of almost 190 km, with monitoring station depths ranging from 27 to 1465 m. In addition to temperature, salinity and oxygen CTD profile data, archived monitoring data for the macro-nutrients (phosphate, nitrate + nitrite, silicate) and chlorophyll-a was evaluated. The climatologies exhibit clear spatial and seasonal variability patterns for all parameters, that yield important insight into the SBUS upwelling cycle. These data sets comprise valuable additions to our knowledge base, and will aid both future modelling efforts and studies of biogeochemical processes in upwelling systems. Data for the constructed climatologies have been made available via the PANGAEA Data Archiving and Publication database at http://doi.pangaea.de/10.1594/PANGAEA.882218.

Download Full-text

DATA ARCHIVING AND DISTRIBUTION OF LiDAR AND DERIVED DATASETS IN THE PHILIPPINES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b4-195-2016 ◽

2016 ◽

Vol XLI-B4 ◽

pp. 195-199

Author(s):

M. E. A. Tupas ◽

S. C. Lat ◽

R. A. Magturo

Keyword(s):

Data Acquisition ◽

Spatial Data ◽

Distribution System ◽

Data Exchange ◽

Flood Hazard ◽

Data Access ◽

The Philippines ◽

Data Sets ◽

Data Archiving ◽

Main Site

LiDAR programs in the Philippines have been generating valuable resource and hazard information for most of the country at a substantial rate since 2012. Significant progress have been made due to the programs design of engaging 16 Universities and research institutions spatially distributed across the country. Because of this, data has been accumulating at a brisk rate which poses significant technical and logistic issues. While a central node, the University of the Philippines, Diliman, handles data acquisition, pre-processing, and quality checking, processing and ground validation are devolved to the various nodes. For this setup to be successful, an efficient data access and distribution system should be in place. <br><br> In this paper, we discuss the spatial data infrastructure and data access protocols implemented by the program. At the center of the data access and distribution operations is LiPAD or our LiDAR portal for archiving and distribution. LiPAD is built on open source technologies, established web standards, and protocols. At its back-end a central data archive has been established using state of the art Object Storage technology to store both raw, processed Lidar and derived data sets. Catalog of available data sets ranging from data acquisition foot prints, to DEM coverages, to derived products such as flood hazard, and crop suitability are viewable and accessible on the main site based on the popular GeoNode application. Data exchange is performed using varying protocols to address various logistical problems. Given the various challenges the program is successful in distributing data sets not just to partner processing nodes but to other stakeholders where main requesters include national agencies and general research and academic institutions.

Download Full-text

Locality-Sensitive Hashing for Information Retrieval System on Multiple GPGPU Devices

Applied Sciences ◽

10.3390/app10072539 ◽

2020 ◽

Vol 10 (7) ◽

pp. 2539 ◽

Cited By ~ 1

Author(s):

Toan Nguyen Mau ◽

Yasushi Inoguchi

Keyword(s):

Big Data ◽

Information Retrieval ◽

Retrieval System ◽

Hash Table ◽

Information Retrieval System ◽

Main Memory ◽

Locality Sensitive Hashing ◽

Data Sets ◽

Similar Data ◽

Data Set

It is challenging to build a real-time information retrieval system, especially for systems with high-dimensional big data. To structure big data, many hashing algorithms that map similar data items to the same bucket to advance the search have been proposed. Locality-Sensitive Hashing (LSH) is a common approach for reducing the number of dimensions of a data set, by using a family of hash functions and a hash table. The LSH hash table is an additional component that supports the indexing of hash values (keys) for the corresponding data/items. We previously proposed the Dynamic Locality-Sensitive Hashing (DLSH) algorithm with a dynamically structured hash table, optimized for storage in the main memory and General-Purpose computation on Graphics Processing Units (GPGPU) memory. This supports the handling of constantly updated data sets, such as songs, images, or text databases. The DLSH algorithm works effectively with data sets that are updated with high frequency and is compatible with parallel processing. However, the use of a single GPGPU device for processing big data is inadequate, due to the small memory capacity of GPGPU devices. When using multiple GPGPU devices for searching, we need an effective search algorithm to balance the jobs. In this paper, we propose an extension of DLSH for big data sets using multiple GPGPUs, in order to increase the capacity and performance of the information retrieval system. Different search strategies on multiple DLSH clusters are also proposed to adapt our parallelized system. With significant results in terms of performance and accuracy, we show that DLSH can be applied to real-life dynamic database systems.

Download Full-text

Managing incidental findings and research results in genomic research involving biobanks and archived data sets

Genetics in Medicine ◽

10.1038/gim.2012.23 ◽

2012 ◽

Vol 14 (4) ◽

pp. 361-384 ◽

Cited By ~ 294

Author(s):

Susan M. Wolf ◽

Brittney N. Crock ◽

Brian Van Ness ◽

Frances Lawrenz ◽

Jeffrey P. Kahn ◽

...

Keyword(s):

Incidental Findings ◽

Genomic Research ◽

Data Sets ◽

Research Results ◽

Archived Data

Download Full-text

Gold Standard for macromolecular crystallography diffraction data

IUCrJ ◽

10.1107/s2052252520008672 ◽

2020 ◽

Vol 7 (5) ◽

pp. 784-792

Author(s):

Herbert J. Bernstein ◽

Andreas Förster ◽

Asmit Bhowmick ◽

Aaron S. Brewster ◽

Sandor Brockhauser ◽

...

Keyword(s):

Gold Standard ◽

Three Dimensional ◽

Biological Macromolecules ◽

Light Sources ◽

Data Sets ◽

Macromolecular Crystallography ◽

Data Archiving ◽

High Data ◽

Standard Application ◽

Fair Principles

Macromolecular crystallography (MX) is the dominant means of determining the three-dimensional structures of biological macromolecules. Over the last few decades, most MX data have been collected at synchrotron beamlines using a large number of different detectors produced by various manufacturers and taking advantage of various protocols and goniometries. These data came in their own formats: sometimes proprietary, sometimes open. The associated metadata rarely reached the degree of completeness required for data management according to Findability, Accessibility, Interoperability and Reusability (FAIR) principles. Efforts to reuse old data by other investigators or even by the original investigators some time later were often frustrated. In the culmination of an effort dating back more than two decades, a large portion of the research community concerned with high data-rate macromolecular crystallography (HDRMX) has now agreed to an updated specification of data and metadata for diffraction images produced at synchrotron light sources and X-ray free-electron lasers (XFELs). This `Gold Standard' will facilitate the processing of data sets independent of the facility at which they were collected and enable data archiving according to FAIR principles, with a particular focus on interoperability and reusability. This agreed standard builds on the NeXus/HDF5 NXmx application definition and the International Union of Crystallography (IUCr) imgCIF/CBF dictionary, and it is compatible with major data-processing programs and pipelines. Just as with the IUCr CBF/imgCIF standard from which it arose and to which it is tied, the NeXus/HDF5 NXmx Gold Standard application definition is intended to be applicable to all detectors used for crystallography, and all hardware and software developers in the field are encouraged to adopt and contribute to the standard.

Download Full-text