Polymorphic Compressed Replication of Columnar Data in Scale-Up Hybrid Memory Systems

In-memory database systems adopting a columnar storage model play a crucial role with respect to data analytics. While data is completely kept in-memory by these systems for efficiency, data has to be stored on a non-volatile medium for persistence and fault tolerance as well. Traditionally, slow block-level devices like HDDs or SSDs are used which, however, can be replaced by fast byte-addressable NVRAM nowadays. Thus, hybrid memory systems consisting of DRAM and NVRAM offer a great opportunity for column-oriented database systems to persistently store and to efficiently process columnar data exclusively in main-memory. However, possible DRAM and NVRAM failures still necessitate the protection of primary data. While data replication is a suitable means, it increases the NVRAM endurance problem through increased write activities. To tackle that challenge and to reduce the overhead of replication, we propose a novel Polymorphic Compressed Replication (PCR) mechanism representing replicas using lightweight compression algorithms to reduce NVRAM writes, while supporting different compressed formats for the replicas of one column to facilitate different database operations during query processing. To show the feasibility and applicability, we developed an inmemory column-store prototype transparently employing PCR through an abstract user-space library. Based on this prototype, our conducted experiments show the effectiveness of our proposed PCR mechanism.