Inter-cluster job coordination using mobile agents

Abstract

Cluster computing is becoming increasingly popular among users of parallel and distributed applications. However since few clusters are solely dedicated to run individual user jobs, it is necessary to coordinate those jobs among independently administered clusters effectively and with less user interaction. For this purpose, we have developed an infrastructure for inter-cluster job coordination using Voyager mobile agents. Once a user submits a job request, which is converted into XML, a mobile agent searches for the most available cluster establishes a job execution environment, executes the job, and reports its results back to the client. Using Voyager's dynamic aggregation feature, we also plan to make such job coordination agents to evolve themselves as to be suitable to be deployed in dynamic cluster environments. This paper presents the design principles and the latest implementation status of our infrastructure as well as advanced services using evolvable agents.