Light-Weight Synthesis of Security Logs for Evaluation of Anomaly Detection and Security Related Experiments

Abstract

Recent decades saw the development of a plethora of approaches that aim to use artificial intelligence to detect anomalies and potential signs of compromise in a computer network. These approaches have commonly been trained and evaluated using only a small number of datasets, which were often criticised in literature. Developing new datasets for this purpose tends to be very resource consuming, as they usually rely on testbeds and network emulation. While this level of details is important for anomaly detection over network traffic, which inspects details of network packets, it is superfluous in cases when such algorithms work with logs of security controls, such as in SIEM systems and approaches for alert correlation. Moreover, evaluation over a testbed generated dataset may not be relevant for the target IT system. In this paper, we propose a light-weight method to enrich existing security control logs with carefully crafted synthetic records that would be produced in case of cyber attacks. This method does not require running a dedicated testbed or comparable specialized equipment. We prepare a set of attack records with emphasis on network scans, and perform experiments with real-world firewall logs and several common anomaly detection algorithms to demonstrate that the injected records are appropriately integrated into the original logs. In the end, we propose future experiments to properly validate the quality of the datasets produced using the proposed method.