Improving GridFTP performance using the Phoebus session layer

Abstract

Phoebus is an infrastructure for improving end-to-end throughput in high-bandwidth, long-distance networks by using a "session layer" protocol and "gateways" in the network. Phoebus has the ability to dynamically allocate network resources and to use segmentspecific transport protocols between gateways, as well as to apply other performance-improving techniques on behalf of the user. One of the key data movement applications in high-performance and Grid computing is GridFTP from the Globus project. GridFTP features a modular library interface called XIO that allows it to use alternative transport mechanisms. To facilitate use of the Phoebus system, we have implemented a Globus XIO driver for Phoebus. This paper presents tests of the Phoebus-enabled GridFTP over a network testbed that allows us to modify latency and loss rates. We discuss use of various transport connections, both end-to-end and hop-by-hop, and evaluate the performance of a variety of cases. We demonstrate that Phoebus can easily improve performance in a diverse set of scenarios and of cases, in many instance it outperforms the state of the art.