SCOUT: A Monitor and Profiler of Grid Resources for Large-Scale Scientific Computing

Abstract

Computational Grids consist of heterogeneous collections of geographically distributed computing resources and have supported numerous scientific applications that require substantial amounts of computing power and storage space. From the point of view of scientists who want to leverage these Grid computing resources, effectively locating appropriate computing resources with minimized allocation overheads is crucial to successfully execute large-scale scientific applications. However, Grid resource availability is highly unstable and current Grid Information Service (GIS) does not provide accurate state information of computing resources. This can make it very difficult for users and systems (Schedulers, Resource brokers) to schedule the jobs in the Grid system and to map tasks on appropriate available resources. In this paper, we present SCOUT system that can provide scientific users with current state information about Grid computing resources including the number of available CPU cores and average response time to get resources allocated. With the help of SCOUT, we can periodically profile resource availability of the Computing Elements (CE) in Grids and monitor their average response time and performance. It provides a mechanism to find out the number of available CPU cores required for the applications to execute their tasks within shortest expected time which can accelerate the productivity of leveraging Grid computing resources for solving complex and challenging scientific problems. We have performed resource profiling based on SCOUT system on two different VO(Virtual Organization)s during one month period and based on that information, we could successfully perform large-scale drug repositioning simulations over 2,000 CPU cores.