Transparent Optimization of Grid Server Selection With Real-Time Passive Network Measurements

Abstract

Grid services have tremendously simplified the programming challenges in leveraging large-scale distributed computing. At the same time, the increased level of abstraction reduces the opportunities available to the application for optimizing its performance by monitoring the system. In this paper we introduce a monitoring grid services proxy, which transparently monitors network performance and selects between several replica service providers. This approach provides optimized server selection without any modification to or even awareness of the client application or service providers. We describe how we implement the proxy and monitor the available bandwidth to the service providers using the Wren monitoring toolkit. We present analysis indicating that our monitoring has negligible overhead. Finally, we demonstrate the practicality of our approach by optimizing the server selection for INCOGEN's VIBE, a bioinformatics workflow application that uploads gene sequences for analysis by remote service providers.