QuickAdapt: Scalable Adaptation for Big Data Cyber Security Analytics

Abstract

Big Data Cyber Security Analytics (BDCA) leverages big data technologies for collecting, storing, and analyzing a large volume of security events data to detect cyber-attacks. Accuracy and response time, being the most important quality concerns for BDCA, are impacted by changes in security events data. Whilst it is promising to adapt a BDCA system's architecture to the changes in security events data for optimizing accuracy and response time, it is important to consider large search space of architectural configurations. Searching a large space of configurations for potential adaptation incurs an overwhelming adaptation time, which may cancel the benefits of adaptation. We present an adaptation approach, QuickAdapt, to enable quick adaptation of a BDCA system. QuickAdapt uses descriptive statistics (e.g., mean and variance) of security events data and fuzzy rules to (re) compose a system with a set of components to ensure optimal accuracy and response time. We have evaluated QuickAdapt for a distributed BDCA system using four datasets. Our evaluation shows that on average QuickAdapt reduces adaptation time by 105× with a competitive adaptation accuracy of 70% as compared to an existing solution.