Attack structure matters: Causality-preserving metrics for Provenance-based Intrusion Detection Systems

Provenance-based Intrusion Detection Systems (PIDS) detect attacks and reconstruct attack scenarios by analyzing provenance graphs. These graphs, constructed from events captured by system logs and security sensors, model the causal relationships between operations performed by system entities. In PIDS research, evaluations typically rely on standard metrics such as precision and recall, computed at the graph level. To assess the accuracy of reconstructed attack graphs, researchers often use proxy metrics at the node level, as computing similarity between provenance graphs remains an open problem. In this paper, we address this problem by introducing SDTED (Structure and Depth Preserving Tree Edit Distance), a variant of the recently proposed Generalized Weisfeiler–Lehman Graph Kernel, adapted to capture the distinctive properties of provenance graphs. Using a dataset of attack scenarios from the DARPA Engagements program, we show that SDTED accurately measures similarity between provenance graphs in cases where node-level metrics yield suboptimal results. Moreover, SDTED is capable of detecting changes in causal relationships between provenance graphs, an essential property for robust evaluation of PIDS proposals. We open source our implementation of SDTED to support reproducibility and encourage adoption within the research community.