Position: short object lifetimes require a delete-optimized storage system

Abstract

Early file systems were designed with the expectation that data would typically be read from disk many times before being deleted; on-disk structures were therefore optimized for reading. As main memory sizes increased, more read requests could be satisfied from data cached in memory, motivating file system designs that optimize write performance. Here, we describe how one might build a storage system that optimizes not only reading and writing, but creation and deletion as well. Efficiency is achieved, in part, by automating deletion based on relative retention values rather than requiring data be deleted explicitly by an application. This approach is well suited to an emerging class of applications that process data at consistently high rates of ingest. This paper explores trade-offs in clustering data by retention value and age and examines the effects of allowing the retention values to change under application control.