GridFactory: Distributed Computing on Ephemeral Resources

Abstract

A novel batch system for high throughput computing is presented. The system is specifically designed to leverage virtualization and web technology to facilitate deployment on cloud and other ephemeral resources. In particular, it implements a security model suited for forming collaborations on demand, across organizational boundaries. Other features include firewall-friendliness, integrated software provisioning and virtual machine management. We describe the design and architecture of Grid Factory and four applications that utilize it. Moreover, we compare the performance of Grid Factory in different deployment scenarios.