Log filtering and interpretation for root cause analysis

Problem diagnosis in large software systems is a challenging and complex task. The sheer complexity and size of the logged data make it often difficult for human operators and administrators to perform problem diagnosis and root cause analysis. A challenge in this area is to provide the necessary means, tools, and techniques for the operators to focus their attention to specific parts of the logged data reducing thus the complexity of the diagnostic process. In this paper, we propose a framework for filtering logs according to specific analysis goals and diagnostic hypotheses set by the user or by an automated process. More specifically, the proposed framework uses annotated goal trees to model the constraints and the conditions by which the functionality of a particular system is being delivered. Next, a transformation process maps such constraints and conditions to a collection of queries that can be either applied to a relational database that stores the logged data or use Latent Semantic Indexing to identify the most relevant log entries for the given query. The results of such queries provide a subset of the logged data that is compliant with the goal tree and can be used by a diagnostic SAT-solver based algorithm. Experimental results show that the filtering process can reduce the time and complexity of the diagnosis when applied to multitier heterogeneous service oriented systems.