Realistic Modeling of Resources for a Large-Scale Computational Grid System

2012 ◽  
Vol 4 (3) ◽  
pp. 1-29
Author(s):  
Uei-Ren Chen ◽  
Yun-Ching Tang
Keyword(s):  
Large Scale ◽  
Simulated Data ◽  
Computational Grid ◽  
Computing Environment ◽  
Resource Model ◽  
Grid Resource ◽  
Normal Distributions ◽  
Realistic Modeling ◽  
Source Data ◽  
Grid Resources

To build a large-scale computational grid resource model with realistic characteristics, this paper proposes a statistical remedy to approximate the distributions of computational and communicational abilities of resources. After fetching the source data of resources including computing devices and networks from the real world, the gamma and normal distributions are employed to approximate probabilities for the ability distribution of grid resources. With this method, researchers can supply the computational grid simulators with required parameters relating to computational and communicational abilities. The experiment result shows that proposed methodology can lead to a better precision according to the error measurement of the simulated data and its source. The proposed method can support existing modern grid simulators with a high-precision resource model which can consider the characteristics of distribution for its computational and communicational abilities in the grid computing environment.

Reliability Based Scheduling Model (RSM) for Computational Grids

2012 ◽  
pp. 37-54
Author(s):  
Zahid Raza ◽  
Deo P. Vidyarthi
Keyword(s):  
Data Exchange ◽  
Large Scale ◽  
A Priori ◽  
Dynamic Environment ◽  
Computational Grid ◽  
Computational Grids ◽  
Scheduling Model ◽  
Reliability Computation ◽  
Grid Resources ◽  
Maximum Reliability

Computational Grid attributed with distributed load sharing has evolved as a platform to large scale problem solving. Grid is a collection of heterogeneous resources, offering services of varying natures, in which jobs are submitted to any of the participating nodes. Scheduling these jobs in such a complex and dynamic environment has many challenges. Reliability analysis of the grid gains paramount importance because grid involves a large number of resources which may fail anytime, making it unreliable. These failures result in wastage of both computational power and money on the scarce grid resources. It is normally desired that the job should be scheduled in an environment that ensures maximum reliability to the job execution. This work presents a reliability based scheduling model for the jobs on the computational grid. The model considers the failure rate of both the software and hardware grid constituents like application demanding execution, nodes executing the job, and the network links supporting data exchange between the nodes. Job allocation using the proposed scheme becomes trusted as it schedules the job based on a priori reliability computation.

Automatic Partitioning of Large Scale Simulation in Grid Computing for Run Time Reduction

2010 ◽  
Vol 1 (2) ◽  
pp. 64-90 ◽  
Author(s):  
Nurcin Celik ◽  
Esfandyar Mazhari ◽  
John Canby ◽  
Omid Kazemi ◽  
Parag Sarfare ◽  
...  
Keyword(s):  
Execution Time ◽  
Large Scale ◽  
Time Synchronization ◽  
Computational Grid ◽  
Time Interval ◽  
Computational Power ◽  
Computing Environment ◽  
Large Scale Systems ◽  
Large Scale Simulations ◽  
Reduce Execution Time

Simulating large-scale systems usually entails exhaustive computational powers and lengthy execution times. The goal of this research is to reduce execution time of large-scale simulations without sacrificing their accuracy by partitioning a monolithic model into multiple pieces automatically and executing them in a distributed computing environment. While this partitioning allows us to distribute required computational power to multiple computers, it creates a new challenge of synchronizing the partitioned models. In this article, a partitioning methodology based on a modified Prim’s algorithm is proposed to minimize the overall simulation execution time considering 1) internal computation in each of the partitioned models and 2) time synchronization between them. In addition, the authors seek to find the most advantageous number of partitioned models from the monolithic model by evaluating the tradeoff between reduced computations vs. increased time synchronization requirements. In this article, epoch- based synchronization is employed to synchronize logical times of the partitioned simulations, where an appropriate time interval is determined based on the off-line simulation analyses. A computational grid framework is employed for execution of the simulations partitioned by the proposed methodology. The experimental results reveal that the proposed approach reduces simulation execution time significantly while maintaining the accuracy as compared with the monolithic simulation execution approach.

Implementation of Grid Resources and Services Based on WSRF

2011 ◽  
Vol 225-226 ◽  
pp. 642-645
Author(s):  
Luo Zhong ◽  
Gan Zuo ◽  
Hua Zhu Song
Keyword(s):  
Web Service ◽  
Standard Protocol ◽  
Resource Model ◽  
Grid Resource ◽  
Service Resource ◽  
Grid Resources ◽  
Implementation Steps

Traditional web service resource hasn't the concept of state and exists many limitations in application. WS-Resource of WSRF solved this problem. This paper elaborated the principal standard protocol specifications in WSRF, analyzed WS-resources characteristics, gave the implementation steps of grid resource and service, and finally validated the superiority of WSRF resource model with an example.

Reliability Based Scheduling Model (RSM) for Computational Grids

2011 ◽  
Vol 2 (2) ◽  
pp. 20-37 ◽  
Author(s):  
Zahid Raza ◽  
Deo P. Vidyarthi
Keyword(s):  
Data Exchange ◽  
Large Scale ◽  
A Priori ◽  
Dynamic Environment ◽  
Computational Grid ◽  
Computational Grids ◽  
Scheduling Model ◽  
Reliability Computation ◽  
Grid Resources ◽  
Maximum Reliability

Computational Grid attributed with distributed load sharing has evolved as a platform to large scale problem solving. Grid is a collection of heterogeneous resources, offering services of varying natures, in which jobs are submitted to any of the participating nodes. Scheduling these jobs in such a complex and dynamic environment has many challenges. Reliability analysis of the grid gains paramount importance because grid involves a large number of resources which may fail anytime, making it unreliable. These failures result in wastage of both computational power and money on the scarce grid resources. It is normally desired that the job should be scheduled in an environment that ensures maximum reliability to the job execution. This work presents a reliability based scheduling model for the jobs on the computational grid. The model considers the failure rate of both the software and hardware grid constituents like application demanding execution, nodes executing the job, and the network links supporting data exchange between the nodes. Job allocation using the proposed scheme becomes trusted as it schedules the job based on a priori reliability computation.

scholarly journals Enhancement of Detecting Permanent Water and Temporary Water in Flood Disasters by Fusing Sentinel-1 and Sentinel-2 Imagery Using Deep Learning Algorithms: Demonstration of Sen1Floods11 Benchmark Datasets

2021 ◽  
Vol 13 (11) ◽  
pp. 2220
Author(s):  
Yanbing Bai ◽  
Wenqi Wu ◽  
Zhengxin Yang ◽  
Jinze Yu ◽  
Bo Zhao ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Large Scale ◽  
Learning Algorithms ◽  
Flood Events ◽  
Test Set ◽  
Source Data ◽  
Flood Disasters ◽  
Temporary Water ◽  
Sentinel 2

Identifying permanent water and temporary water in flood disasters efficiently has mainly relied on change detection method from multi-temporal remote sensing imageries, but estimating the water type in flood disaster events from only post-flood remote sensing imageries still remains challenging. Research progress in recent years has demonstrated the excellent potential of multi-source data fusion and deep learning algorithms in improving flood detection, while this field has only been studied initially due to the lack of large-scale labelled remote sensing images of flood events. Here, we present new deep learning algorithms and a multi-source data fusion driven flood inundation mapping approach by leveraging a large-scale publicly available Sen1Flood11 dataset consisting of roughly 4831 labelled Sentinel-1 SAR and Sentinel-2 optical imagery gathered from flood events worldwide in recent years. Specifically, we proposed an automatic segmentation method for surface water, permanent water, and temporary water identification, and all tasks share the same convolutional neural network architecture. We utilize focal loss to deal with the class (water/non-water) imbalance problem. Thorough ablation experiments and analysis confirmed the effectiveness of various proposed designs. In comparison experiments, the method proposed in this paper is superior to other classical models. Our model achieves a mean Intersection over Union (mIoU) of 52.99%, Intersection over Union (IoU) of 52.30%, and Overall Accuracy (OA) of 92.81% on the Sen1Flood11 test set. On the Sen1Flood11 Bolivia test set, our model also achieves very high mIoU (47.88%), IoU (76.74%), and OA (95.59%) and shows good generalization ability.

scholarly journals Constructing Large-Scale Genetic Maps Using an Evolutionary Strategy Algorithm

Genetics ◽  
2003 ◽  
Vol 165 (4) ◽  
pp. 2269-2282
Author(s):  
D Mester ◽  
Y Ronin ◽  
D Minkov ◽  
E Nevo ◽  
A Korol
Keyword(s):  
Discrete Optimization ◽  
High Performance ◽  
Large Scale ◽  
Simulated Data ◽  
Real Data ◽  
Genetic Maps ◽  
Chromosome 1 ◽  
Evolutionary Strategy ◽  
Group A ◽  
The One

Abstract This article is devoted to the problem of ordering in linkage groups with many dozens or even hundreds of markers. The ordering problem belongs to the field of discrete optimization on a set of all possible orders, amounting to n!/2 for n loci; hence it is considered an NP-hard problem. Several authors attempted to employ the methods developed in the well-known traveling salesman problem (TSP) for multilocus ordering, using the assumption that for a set of linked loci the true order will be the one that minimizes the total length of the linkage group. A novel, fast, and reliable algorithm developed for the TSP and based on evolution-strategy discrete optimization was applied in this study for multilocus ordering on the basis of pairwise recombination frequencies. The quality of derived maps under various complications (dominant vs. codominant markers, marker misclassification, negative and positive interference, and missing data) was analyzed using simulated data with ∼50-400 markers. High performance of the employed algorithm allows systematic treatment of the problem of verification of the obtained multilocus orders on the basis of computing-intensive bootstrap and/or jackknife approaches for detecting and removing questionable marker scores, thereby stabilizing the resulting maps. Parallel calculation technology can easily be adopted for further acceleration of the proposed algorithm. Real data analysis (on maize chromosome 1 with 230 markers) is provided to illustrate the proposed methodology.

scholarly journals Cloud-based intelligent self-diagnosis and department recommendation service using Chinese medical BERT

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Junshu Wang ◽  
Guoming Zhang ◽  
Wei Wang ◽  
Ka Zhang ◽  
Yehua Sheng
Keyword(s):  
Cloud Computing ◽  
Large Scale ◽  
Medical Service ◽  
Rapid Development ◽  
Medical Knowledge ◽  
Language Models ◽  
Computing Environment ◽  
Computing Power ◽  
Cloud Computing Environment ◽  
Proposed Model

AbstractWith the rapid development of hospital informatization and Internet medical service in recent years, most hospitals have launched online hospital appointment registration systems to remove patient queues and improve the efficiency of medical services. However, most of the patients lack professional medical knowledge and have no idea of how to choose department when registering. To instruct the patients to seek medical care and register effectively, we proposed CIDRS, an intelligent self-diagnosis and department recommendation framework based on Chinese medical Bidirectional Encoder Representations from Transformers (BERT) in the cloud computing environment. We also established a Chinese BERT model (CHMBERT) trained on a large-scale Chinese medical text corpus. This model was used to optimize self-diagnosis and department recommendation tasks. To solve the limited computing power of terminals, we deployed the proposed framework in a cloud computing environment based on container and micro-service technologies. Real-world medical datasets from hospitals were used in the experiments, and results showed that the proposed model was superior to the traditional deep learning models and other pre-trained language models in terms of performance.

A Heuristic Resource Management Method in Grid Computing Environment

2011 ◽  
Vol 50-51 ◽  
pp. 521-525
Author(s):  
Xian Mei Fang
Keyword(s):  
Resource Management ◽  
Grid Computing ◽  
Collaborative Research ◽  
Ant Colony Optimization Algorithm ◽  
Computing Environment ◽  
Grid Resource ◽  
Grid Environment ◽  
Load Imbalance ◽  
The World ◽  
Management Method

Grid is an emerging infrastructure which enables effective coordinate access to various distributed computing resources in order to serve the needs of collaborative research and work across the world. Grid resource management is always a key subject in the grid computing. We first analyze the resource management in the grid computing environment, then according to the load imbalance question in the ant colony optimization algorithm, propose an improved algorithm that suits to be used in the grid environment.

Grid Resource Allocation with Genetic Algorithm Using Population Based on Multisets

2017 ◽  
Vol 26 (1) ◽  
pp. 169-184 ◽  
Author(s):  
Absalom E. Ezugwu ◽  
Nneoma A. Okoroafor ◽  
Seyed M. Buhari ◽  
Marc E. Frincu ◽  
Sahalu B. Junaidu
Keyword(s):  
Genetic Algorithm ◽  
Resource Allocation ◽  
Grid Computing ◽  
Resource Selection ◽  
Computing System ◽  
Population Based ◽  
Fine Tuning ◽  
Grid Resource ◽  
Performance Results ◽  
Grid Resources

AbstractThe operational efficacy of the grid computing system depends mainly on the proper management of grid resources to carry out the various jobs that users send to the grid. The paper explores an alternative way of efficiently searching, matching, and allocating distributed grid resources to jobs in such a way that the resource demand of each grid user job is met. A proposal of resource selection method that is based on the concept of genetic algorithm (GA) using populations based on multisets is made. Furthermore, the paper presents a hybrid GA-based scheduling framework that efficiently searches for the best available resources for user jobs in a typical grid computing environment. For the proposed resource allocation method, additional mechanisms (populations based on multiset and adaptive matching) are introduced into the GA components to enhance their search capability in a large problem space. Empirical study is presented in order to demonstrate the importance of operator improvement on traditional GA. The preliminary performance results show that the proposed introduction of an additional operator fine-tuning is efficient in both speed and accuracy and can keep up with high job arrival rates.

Sign in / Sign up

Export Citation Format

Share Document