MidLog: An automated log anomaly detection method based on multi-head GRU

Abstract

Software systems typically utilize logs to record events that contain critical information. These logs are an indispensable data source for analyzing system anomalies. Large-scale log datasets have placed a tremendous burden on manually analyzing system logs as it is extremely time-consuming and error-prone. There have been many studies on log anomaly detection, whereas most existing deep learning methods lack flexibility and need auxiliary features to improve detection accuracy. We propose an automated anomaly detection method based on multi-head GRU for system logs, called MidLog. The core idea comes from the multi-head mechanism in Transformer. Multiple GRUs are used to learn normal sequence patterns hidden in system logs. Each GRU network is only responsible for learning a local sequence pattern. We conduct a global analysis of these local patterns to achieve log anomaly detection, which facilitates more accurate identification of log anomalies. The number of base models (GRUs) can be easily increased or decreased under the multi-head mechanism. Such a characteristic gives MidLog more flexibility and allows for a trade-off between detection accuracy and efficiency. Experiment results on public log datasets show that our method can achieve better detection accuracy compared with baseline methods.