Evaluation of the data handling pipeline of the ASTRID framework

Abstract

Effective attack detection and security analytics rely on the availability of timely and fine-grained information about the evolving context of the protected environment. The data handling process entails collection from heterogeneous sources, local aggregation and transformation operations before transmission, and finally collection and delivery to multiple processing engines for analysis and correlation. Many Security Information and Event Management (SIEM) tools work according to the “funnel” principle: gather as much data as possible and then filter it to keep the relevant information. However, this might lead to unacceptable overhead, especially when monitoring containerized environments. As part of our activity in ASTRID, we therefore conducted experimental investigation on resource consumption of the data handling pipeline, starting from embedded agents up to delivery to the Context Broker.