DMA-Assisted I/O for Persistent Memory

Abstract

Modern local persistent memory (PM) file systems often rely on CPU-based memory copying for data transfer between DRAM and PM, resulting in significant CPU resource consumption. While some nascent systems explore DMA (direct memory access) as an alternative for improved efficiency, the intricacies and trade-offs remain obscure. This paper investigates the feasibility of DMA for PM I/O and argues that it is not a straightforward replacement for CPU-based methods. Two key limitations hinder the direct adoption: poor performance for small data and limited bandwidth. To relieve these issues, we propose PM-DMA, a novel I/O mechanism that leverages the strengths of both CPU and DMA. It incorporates three key components: (1) L-Switch, seamlessly switches between CPU and DMA modes based on workload characteristics, maximizing performance; (2) D-Pool, reduces DMA setup overhead, improving responsiveness; (3) P-Mode, allows servicing requests through multiple channels, even hybrid CPU-DMA ones, for enhanced throughput. We implemented PM-DMA on two well-known PM file systems, NOVA and WineFS, utilizing Intel I/OAT technology. Our experimental results demonstrate substantial CPU consumption reductions across diverse workloads. Notably, under heavy load, PM-DMA delivers up to a $10.4\times$ performance improvement.