Provision of Disk I/O Guarantee for MapReduce Applications

Abstract

YARN is an emerging resource management in the Hadoop ecosystem, where big data in the scale of petabytes/day are processed with the use of commercial off-the-shelf servers. At present YARN supports only RAM and CPU reservation/control. However, the reservation and the control of disk I/O throughput are also needed to provide a satisfactory performance for MapReduce applications. In this paper, we propose a solution with software components that can be integrated into YARN to support the reservation of disk I/O throughput in YARN. Experiments show that a proposed solution can provide a disk I/O throughput guarantee for MapReduce applications.