Provisioning a Multi-tiered Data Staging Area for Extreme-Scale Machines

2011 ◽  
Author(s):  
Ramya Prabhakar ◽  
Sudharshan S. Vazhkudai ◽  
Youngjae Kim ◽  
Ali R. Butt ◽  
Min Li ◽  
...  
Keyword(s):  
Data Staging ◽  
Staging Area ◽  
Extreme Scale

Optimizing ETL by a Two-Level Data Staging Method

2016 ◽  
Vol 12 (3) ◽  
pp. 32-50
Author(s):  
Xiufeng Liu ◽  
Nadeem Iftikhar ◽  
Huan Huo ◽  
Per Sieverts Nielsen
Keyword(s):  
Data Warehouse ◽  
Data Warehousing ◽  
Data Extraction ◽  
Data Staging ◽  
Staging Area ◽  
Level Data ◽  
Different Types ◽  
Operational Systems ◽  
Early Late ◽  
Central Data

In data warehousing, the data from source systems are populated into a central data warehouse (DW) through extraction, transformation and loading (ETL). The standard ETL approach usually uses sequential jobs to process the data with dependencies, such as dimension and fact data. It is a non-trivial task to process the so-called early-/late-arriving data, which arrive out of order. This paper proposes a two-level data staging area method to optimize ETL. The proposed method is an all-in-one solution that supports processing different types of data from operational systems, including early-/late-arriving data, and fast-/slowly-changing data. The introduced additional staging area decouples loading process from data extraction and transformation, which improves ETL flexibility and minimizes intervention to the data warehouse. This paper evaluates the proposed method empirically, which shows that it is more efficient and less intrusive than the standard ETL method.

scholarly journals Extract Transform Load (ETL) Process in Distributed Database Academic Data Warehouse

2019 ◽  
Vol 4 (2) ◽  
pp. 61-68
Author(s):  
Ardhian Agung Yulianto
Keyword(s):  
Decision Making ◽  
Data Warehouse ◽  
Distributed Database ◽  
Data Sources ◽  
Data Delivery ◽  
Multidimensional Model ◽  
Data Staging ◽  
Staging Area ◽  
Data Source ◽  
Sql Query

While data warehouse is designed to support the decision-making function, the most time-consuming part is Extract Transform Load (ETL) process. Case in Academic Data Warehouse, when data source came from faculty’s distributed database, although having a typical database but become not easier to integrate. This paper presents the ETL detail process following Data Flow Thread in data staging area for identifying, profiling, the content analyzing including all tables in data sources, and then cleaning, confirming dimension and data delivery to the data warehouse. Those processes are running gradually from each distributed database data sources until it merged. Dimension table and fact table are generated in a multidimensional model. ETL tool is Pentaho Data Integration 6.1. ETL testing is done by comparing data source and data target and DW testing conducted by comparing the data analysis between SQL query and Saiku Analytics plugin in Pentaho Business Analytic Server.

Evidence of Brent Geese Branta bernicla Moving between an Autumn Staging Area in East Hokkaido and Wintering Sites in West Hokkaido and Northern Honshu

2020 ◽  
Vol 19 (2) ◽  
pp. 211
Author(s):  
Yusuke Sawa ◽  
Chieko Tamura ◽  
Toshio Ikeuchi ◽  
Tetsuo Shimada ◽  
Kaoru Fujii ◽  
...  
Keyword(s):  
Branta Bernicla ◽  
Staging Area

Cardiovascular flow simulation at extreme scale

2009 ◽  
Vol 46 (1) ◽  
pp. 71-82 ◽  
Author(s):  
Min Zhou ◽  
Onkar Sahni ◽  
H. Jin Kim ◽  
C. Alberto Figueroa ◽  
Charles A. Taylor ◽  
...  
Keyword(s):  
Flow Simulation ◽  
Cardiovascular Flow ◽  
Extreme Scale

scholarly journals PANDA: Extreme Scale Parallel K-Nearest Neighbor on Distributed Architectures

2016 ◽  
Author(s):  
Md. Mostofa Ali Patwary ◽  
Nadathur Rajagopalan Satish ◽  
Narayanan Sundaram ◽  
Jialin Liu ◽  
Peter Sadowski ◽  
...  
Keyword(s):  
Nearest Neighbor ◽  
K Nearest Neighbor ◽  
Distributed Architectures ◽  
Extreme Scale
Sign in / Sign up

Export Citation Format

Share Document