Clustering Evolving Batch System Jobs for Online Anomaly Detection

Abstract

In batch systems monitoring information at the level of individual jobs is crucial to optimize resource utilization and prevent misusage. However, especially the usage of network resources is difficult to track. In order to understand usage patterns in modern computing clusters, a more detailed monitoring than existent solutions is required. A monitoring on job level leads to dynamic graphs of processes with attached time series data of e.g. network resource usage. Utilizing clustering, common usage patterns can be identified and outliers detected. This work provides an overview about ongoing efforts to cluster dynamic graphs in the context of distributed streams of monitoring events.