Replication-Based Highly Available Metadata Management for Cluster File Systems

Abstract

In cluster file systems, the metadata management is critical to the whole system. Past researches mainly focus on journaling which alone is not enough to provide high-available metadata service. Some others try to use replication, but the extra latency accompanied is a main problem. To guarantee both availability and efficiency, we propose a mechanism for building highly available metadata servers based on replication, which integrates Paxos algorithm effectively into metadata service. The Packed Multi-Paxos is proposed to reduce the latency brought by replication, which is self-adaptive and can make the replication to achieve high throughput under heavy client load and low latency under light client load. By designing efficient architecture and coordination mechanism, all replica server nodes simultaneously provide metadata read-access service. This high-available mechanism could decrease the impact of server failures and there is no interruption of service. The performance results show that the latency caused by replication and redundancy is well under control, and the performance of metadata read operation gains improvement.