SAFE: Structure-aware file and email deduplication for cloud-based storage systems

Abstract

Cloud-based storages have become considerably popular in recent years, as they enable data access from anywhere and any device at any time. Many leading cloud-based storage services including Dropbox, JustCloud, and Mozy use data deduplication techniques at a source to save network bandwidth from a user to cloud servers as well as storage space, which in turn expedites the speed of data upload. Although traditional variable-size block-level deduplication techniques tend to achieve a high data reduction rate, they require a high processing overhead due to data chunking, index processing, and data fragmentation. However, a user's device can be limited in processing capability and memory space to perform an effective client side deduplication. While, a simple file-level or a large fixed-size block-level deduplication may be able to cope with the limited source device capacity, it cannot produce a high data reduction rate. In this paper, we propose a novel Structure-Aware File and Email deduplication (SAFE) scheme that achieves both fast and effective data reduction for cloud-based storage services. SAFE efficiently deduplicates redundant objects in structured files as well as emails exploiting object-level components based on their structures. Our evaluation using real data sets of structured files and emails shows that SAFE accomplishes as good of storage savings as a variable-block deduplication, while being as fast as a file-level or a large fixed-size block-level deduplication.