Creating operational profiles of software systems by transforming their log files to directed cyclic graphs

Abstract

Most log files are of one format - a flat file with the events of execution recorded one after the other. Each line in the file contains at least a timestamp, a combination of one or more event identifiers, and the actual log message with information of which event was executed and what the values for the dynamic parameters of that event are. Since log files have this trace information, we can use it for many purposes, such as operational profiling and anomalous execution path detection. However the current flat file format of a log file is very unintuitive to detect the existence of a repeating pattern. In this paper we propose a transformation of the current serial order format of a log file to a directed cyclic graph (such as a non-finite state machine) format and how the operational profile of a system can be built from this representation of the log file. We built a tool (in C++), that transforms a log file with a set of log events in a serial order to an adjacency matrix for the resulting graphical representation. We can then easily apply existing graph theory based algorithms on the adjacency matrix to analyze the log file of the system. The directed cyclic graph and the analysis of it can be visualized by rendering the adjacency matrix with graph visualization tools, like Graphviz.