Parity Redundancy in a Clustered Storage System

Abstract

Distributed storage systems must provide highly available access to data while maintaining high performance and maximum scalability. In addition, reliability in a storage system is of the utmost importance and the correctness and availability of data must be guaranteed. Adding parity redundancy to distributed storage systems has been problematic because of the impact on performance. In this paper, we investigate mechanisms to add redundancy to the Lustre cluster file system with minimal effect on overall system performance. With data spread across multiple nodes, ensuring the consistency of the data requires special techniques. We describe fault tolerant algorithms to maintain the consistency and reliability of the data. We show how these techniques guarantee data integrity and availability of systems for read and write even under failure mode scenarios.