Why applications are coupled to I/O and how the scenery can be changed

Abstract

The versatility and I/O demanding nature of modern applications, compel them to be operable across various storage platforms. As the platforms may exhibit different characteristics, the applications need to incorporate platform-specific code paths. More demanding applications that require concurrent use of platforms must also implement transactions, parallel streams, error handling routines and data reconstruction strategies. Building this logic is a complicated task that requires I/O expertise, but more importantly, it must be re-implemented for each application. To that end, the application decouples from the platform but is bounded to I/O related code paths. This paper presents a framework (TrIO) to decouple the I/O logic from the application source code. The framework exhibits a domain specific language, that allows the developers to model the I/O logic as a graph of components, without any implementation concerns. The runtime engine takes as input the graph and generates the respective intransit data processing network with transactional and parallel data delivery guarantees. As a proof-of-concept, a distributed filesystem prototype built atop the engine, with graphs that imitate RedHat GlusterFS volumes to define its behavior. Conducted tests show that for volumes that rely on parallelization, our prototype can outperform GlusterFS by a factor of 2.