Using Storage Class Memory Efficiently for an In-memory Database

Abstract

Storage class memory (SCM) is an emerging class of memory devices that are both byte addressable, and persistent. There are many different technologies that can be considered SCM, at different stages of maturity. Examples of such technologies include NVDIMM-N, PCM, SttRAM, Racetrack, FeRAM, and others. Currently, applications rely on storage technologies such as Flash memory and hard disks as the physical media for persistency. SCM behaves differently and has significantly different characteristics than these existing technologies. That means there is a need for a fundamental change in the way we program data persistency in applications to fully realize the potential of this new class of device. Previous work such as [1] focuses on designing a filesystem optimized to run over SCM memory. Other projects such as Mnemosyne [4] provide a general purpose API for applications to use SCM memory. Mnemosyne however, does not provide a transaction mechanism flexible enough to allow different SCM updates spanning different parts of the code to be considered one transaction. Our work is focused on employing minimal changes needed to retrofit an existing key-value store to take advantage of SCM technology. We demonstrate these changes on Redis (REmote DIctionary Server) [3], which is a popular key-value store. We show how Redis can be modified to take advantage of these new abilities by allowing the application to manage its own storage in a unique way. Our approach is to use two types of memory technologies (DRAM and SCM) for different purposes in a single application. To optimize the system data capacity, we keep a minimal dataset in persistent memory, while keeping metadata (such as indexing) in DRAM, which can be rebuilt upon failures. Persistency in Redis is currently performed by logging all transactions to an append-only log file (AOF). These transactions can then be replayed to recover from a failure. The transactions are not made persistent until the AOF file is flushed to disk, which is very slow. Flushing after every transaction has a performance impact, but flushing periodically creates a risk of lost data. By using SCM instead of a disk, we can effectively flush every transaction without impacting performance. In order to change Redis to store data objects on the SCM, we must ensure consistency of persistent data even after an unexpected shutdown. To ensure consistency, a modified version of dlmalloc [2] is used for all allocations done on the SCM, and mfence commands are used to overcome unexpected reordering. We model the SCM using a memory mapped file backed on a ramdisk. We compared our changes to Redis using an AOF backed on a ramdisk. Although we don't take into account the latency overheads of accessing the SCM, this comparison gives us a good upper bound of performance benefits of using SCM memory. Our results demonstrate an average latency reduction of 43% and average throughput increase of 75%.