Improving log anomaly detection via spatial pooling: Combining SPClassifier with ensemble method

In the ever-updating field of software development, new bugs emerge daily, requiring significant time for analysis. As a result, research is being conducted on automating bug resolution using techniques such as anomaly detection through deep learning applied to text logs. This study focuses on anomaly detection using text logs and aims to address current challenges. Specifically, we aim to improve the accuracy of SPClassifier, a robust and lightweight AI model capable of handling dynamic log datasets through ad-hoc learning. We employ three ensemble learning methods to enhance the accuracy of SPClassifier. The method that achieved the greatest improvement was Improved Bagging, which combines the non-overlapping sampling of Pasting with the overlapping sampling of Bagging, resulting in a maximum F1-score improvement of 155 %. Additionally, on certain datasets, the F1-score surpassed that of well-known DNN methods by 130 %. Furthermore, the proposed method demonstrated lower variance compared to DNN methods, indicating its advantage, particularly in environments where datasets frequently fluctuate, such as development fields. These results highlight the clear superiority of the proposed method, which is lightweight in terms of computational resources and supports ad-hoc learning.