Multi-core Supported High Performance Security Analytics

Abstract

Such information as system and application logs as well as the output from the deployed security measures, e.g., IDS alerts, firewall logs, scanning reports, etc., is important for the administrators or security operators to be aware at first time of the running state of the system and take efforts if necessary. In this context, high performance security analytics is proposed to address the challenges to rapidly gather, manage, process, and analyze the large amount of real-time information generated from the large scale of enterprise IT-Infrastructure while it is being operated. As an example of next generation Security Information and Event Management (SIEM) platform, Security Analytics Lab (SAL) has been designed and implemented based on the newly emerged In-Memory data management technique, which makes it possible to efficiently organize and access different types of event information through a consistent central storage and interface. To correlate the information from different sources and identify the meaningful information is another challenging task, which makes great sense for quickly judging the current situation and making the decision. In this paper, the multi-core processing technique is introduced in the SAL platform. Various correlation algorithms, e.g., k-means based algorithms, ROCK and QROCK clustering algorithms, have been implemented and integrated in the multi-core supported SAL architecture. Practical experiments are conducted and analyzed to proof that the performance of analytics can be significantly improved by applying multi-core processing technique in SAL.