HEOD: Human-assisted Ensemble Outlier Detection for cybersecurity

Despite extensive academic research in anomaly detection within the cybersecurity domain, its successful adoption in real-world settings remains limited. This paper addresses the challenges of applying outlier detection techniques for threat detection within the context of Security Information and Event Management (SIEM) systems. It particularly highlights the significance of contextualization and explainability, while challenging the assumption that outliers invariably indicate malicious activity. It proposes a simple yet effective outlier detection technique designed to mimic a Security Operation Center (SOC) analyst’s reasoning process in finding anomalies/outliers and deciding maliciousness. The approach emphasizes explainability and simplicity, achieved by combining the output of simple, context-aware univariate submodels that calculate an outlier score for each entry.

The proposed technique is first evaluated on a public dataset, demonstrating its ability to achieve high performance in detecting outliers compared to other well-known algorithms. Furthermore, to assess the practicality in a real-world scenario, the approach is deployed in production alongside the SIEM of a large international enterprise with over 100,000 assets, utilizing 20 terabytes of Endpoint Detection and Response (EDR) logs to detect Living-off-the-Land Binaries (LOLBins). The proposed framework can empower SOC analysts in developing scalable, effective, and interpretable outlier-based threat detection use cases.