Random forest and change point detection for root cause localization in large scale systems

Abstract

Identification of root causes of a performance problem is very difficult in case of large scale IT environment. A model which is scalable and reasonably accurate is required for such complex scenarios. This paper proposes a hybrid model using random forest and statistical change point detection, for root cause localization. Based on impurity measure and change in error rates, random forest identifies the features which can be a potential cause for the problem. Since it is a tree based approach, it does not require any prior information about the measured features. To reduce the number of false classifications, a second level of selection using change point analysis is done. The ability of random forest to work well on very large dataset makes the solution scalable and accurate. Proposed model is applied and verified by identifying the root causes for Service Level Objective Violations in enterprise IT systems.