Improving HDFS performance using local caching system

Abstract

I/O performance of file system in various distributed parallel clusters can critically affect content-aware application performance. In this paper, we present a new architecture for Local Caching System which adjusts content locality between local file system and Hadoop Distributed File System by using proper cache algorithm and managing content queue and location. Also, we show that our local caching system improves the performance of content-aware application sharply.