Group-by Query Process in Middleware of Large Scale Data Intensive Systems

2009 ◽  
Author(s):  
Song Huaiming ◽  
An Mingyuan ◽  
Wang Yang ◽  
Wang Weiping ◽  
Sun Ninghui
Keyword(s):  
Large Scale ◽  
Data Intensive ◽  
Query Process ◽  
Large Scale Data ◽  
Scale Data

scholarly journals Enabling Large-Scale Biomedical Analysis in the Cloud

2013 ◽  
Vol 2013 ◽  
pp. 1-6 ◽  
Author(s):  
Ying-Chih Lin ◽  
Chin-Sheng Yu ◽  
Yen-Jen Lin
Keyword(s):  
High Performance ◽  
Large Scale ◽  
Computing System ◽  
Biomedical Data ◽  
Data Intensive Computing ◽  
Biomedical Analysis ◽  
Data Intensive ◽  
Large Scale Data ◽  
Performance Computing ◽  
Scale Data

Recent progress in high-throughput instrumentations has led to an astonishing growth in both volume and complexity of biomedical data collected from various sources. The planet-size data brings serious challenges to the storage and computing technologies. Cloud computing is an alternative to crack the nut because it gives concurrent consideration to enable storage and high-performance computing on large-scale data. This work briefly introduces the data intensive computing system and summarizes existing cloud-based resources in bioinformatics. These developments and applications would facilitate biomedical research to make the vast amount of diversification data meaningful and usable.

Modified Delay Scheduling: A Heuristic Approach for Hadoop Scheduling to Improve Fairness and Response Time

2015 ◽  
Vol 25 (04) ◽  
pp. 1550009 ◽  
Author(s):  
N. P. Gopalan ◽  
S. Suresh
Keyword(s):  
Response Time ◽  
Large Scale ◽  
Programming Model ◽  
Data Locality ◽  
Data Intensive ◽  
Large Scale Data ◽  
Fair Scheduler ◽  
Hadoop Clusters ◽  
Data Intensive Applications ◽  
Scale Data

Hadoop is a widely used open source implementation of MapReduce which is a popular programming model for parallel processing large scale data intensive applications in a cloud environment. Sharing Hadoop clusters has a tradeoff between fairness and data locality. When launching a local task is not possible, Hadoop Fair Scheduler (HFS) with delay scheduling postpones the node allocation for a while to a job which is to be scheduled next as per fairness to achieve high locality. This waiting becomes waste when the desired locality could not be achieved within a reasonable period. In this paper, a modified delay scheduling in HFS is proposed and implemented in Hadoop. It avoids the aforementioned waiting of delay scheduler if achieving locality is not possible. Instead of blindly waiting for a local node, the proposed algorithm first estimates the time to wait for a local node for the job and avoids waiting wherever achieving locality is not possible within the predefined delay threshold while accomplishing same locality. The performance of the proposed algorithm is evaluated by extensive experiments and it has been observed that the algorithm works significantly better in terms of response time and fairness achieving up to 20% speedup and up to 38% fairness in certain cases.

Sign in / Sign up

Export Citation Format

Share Document