Efficient audit-based compliance for relational data retention

Abstract

The Sarbanes-Oxley Act inspired research on long-term high-integrity retention of business records, leveraging the immutability guarantees that WORM storage servers offer for files. In this paper, we present the transaction log on WORM (TLOW) approach for supporting long-term immutability for relational tuples. TLOW stores the transaction log on WORM and uses an audit helper (AH) add-on to continuously perform audit-related activities without compromising transaction performance or audit trustworthiness. TLOW imposes only 1-11% runtime overhead on TPC-C transactions, much less than previously proposed approaches, and does not require DBMS kernel changes. TLOW audits are extremely fast, e.g., two hours to audit a year of continuous TPC-C activity, versus 10 days for previously proposed approaches. This opens up the possibility of real-time internal audits that can detect fraudulent activity before its effects propagate throughout an enterprise. We also provide a proof of correctness for TLOW, which exposes a subtle threat that affects the correctness of previously proposed approaches.