RepStore: a self-managing and self-tuning storage backend with smart bricks

Abstract

With the continuously improving price-performance ratio, building large, smart-brick based distributed storage system becomes increasingly attractive. The challenges, however, include not only reliability, adequate cost-performance ratio, online upgrades and so on, but also the system's ability to achieve these goals in as self-managing and self-adaptive a manner as possible. In this paper, we describe RepStore, a system that fulfills these goals. RepStore unites the self-organizing capability of P2P DHT and the completely autonomous, per-brick tuning mechanism to derive a scalable and cost-effective architecture. RepStore employs replication for active write-intensive data and erasure-coding for the rest, strives to achieve the best cost-performance balance automatically and transparent to application, and does so in a completely distributed manner. Our preliminary evaluations reveal that the system performs much as expected, achieving performance and reliability closer to a 3-way fully replicated system with only 60% of the cost.