Stretch optimization for virtual screening on multi-user pilot-agent platforms on grid/cloud

Abstract

Virtual screening has proven very effective on grid infrastructures where large scale deployments have led to the identification of active inhibitors for biological targets of interest against malaria, SARS or diabetes. Operating a dedicated virtual screening platform on grid resources requires optimizing the scheduling policy. The scheduling can be done at 2 levels; at site level and at platform level. Site scheduling is done at each site independently; each site is autonomous in its choice of job scheduling. Each site allocates time slots for different groups of users. Platform scheduling is done at group level: inside a time slot jobs from many users are allocated. Pilot agents are sent to sites and act as a container of actual users jobs. They pick up users jobs from a central queue where the second stage scheduling is done. In this paper, we focus on pilot-agent platform shared by many virtual screening users. They need a suitable scheduling algorithm to ensure a certain fairness between users. We have studied the scheduling of users jobs inside central queue and examined the relevance and impact of different scheduling policies (FIFO, SPT, LPT and Round Robin) on the user experience. Optimal criterion used in our research is the stretch, a measure for user experience on the platform. In a first step, we simulated the operation of virtual screening applications on the pilot-agent platform in order to compare the scheduling policies. According to simulation, SPT algorithm was shown to significantly improve scheduling performances. In a second step, the Shortest Processing Time (SPT) and Longest Processing Time (LPT) scheduling policies were implemented on a DIRAC pilot-agent platform at IFI in Hanoi and tested on EGI Biomed Virtual Organization. Experimental results are in good agreement with simulation and confirm that SPT algorithm significantly improves user experience. The relevance of our conclusions also extends to cloud computing. Indeed, cloud infrastructures are also characterized by limited machine availability.