SoKV: Scan performance optimization for KV separation with adaptive dynamic grouping and GC-based LSM-tree management

Abstract

Key-value (KV) storage becomes a foundational technology for system software, enabling fast data processing and high-performance applications across various workloads and scenarios. In KV separation storage systems, where values are stored separately from the LSM-tree, scan operations necessitate traversal of the LSM-tree to retrieve addresses of desired values before accessing the corresponding KV pairs. Consequently, the organization of KV pairs and the size of the LSM-tree significantly impact scan performance. Recognizing this, we devised two strategies: Adaptive Dynamic Grouping and GC-based LSM-tree Management, to enhance scan performance by expediting the restoration of orderliness in frequently accessed KV pairs and reducing LSM-tree size. Finally, we implemented our prototype system called SoKV. Experimental results show that the scan throughput of SoKV is 2.66$\times$ that of RocksDB, 10.38$\times$ that of Parallax, 1.88$\times$ that of WiscKey, 2.27$\times$ that of HashKV and 1.39$\times$ that of FenceKV. Additionally, due to the reduction in the size of the LSM-tree, SoKV also outperforms the all other systems in terms of update performance.