A next generation entropy based framework for alert detection in system logs

Abstract

Recent research efforts have highlighted the capability of entropy based approaches in the automatic discovery of alerts in system logs. In this work, we extend this research to present the evaluations of three entropy based approaches on new datasets not utilized in previous papers. We also extend the approach with the introduction of a Cluster Membership Anomaly score. This extension of the approach is intended to reduce the false positive rates required to detect all alerts. Previous work has shown that false positive rates required for the detection of all alerts for an entropy based approach could be very high. The results show that the Cluster Membership Anomaly score has value for the reduction of false positive rates.