Using Provenance to boost the Metadata Prefetching in distributed storage systems

Abstract

Caching and prefetching are effective approaches to boosting the performance of metadata access in distributed storage systems. Many research efforts have been devoted in developing new metadata prefetching methods by considering past file access patterns. However, the existing methods do not consider the correlations between processes and the corresponding files(e.g. file provenance). Therefore, the methods cannot obtain very rich and accurate correlations, thus decreasing the effectiveness of metadata prefetching. This paper presents a Provenance-based Metadata Prefetching(ProMP) scheme, which considers both provenance and the past file access patterns. Through mining the correlations between processes and corresponding files from provenance and past access history, ProMP can achieve accurate and rich correlation information. ProMP is conducive to employing aggressive metadata prefetching to boost the performance by leveraging the correlations. Our experimental results show that ProMP performs more effectively with less memory overhead than the existing solutions, while improving the hit rates by up to 49% and 7% in contrast to traditional LRU and a state-of-art metadata prefetching algorithm Nexus, respectively.