Data Labeling for Fault Detection in Cloud: A Test Suite-Based Active Learning Approach

Abstract

Ensuring the quality of service of applications deployed in inherently complex and fault-prone cloud environments is of utmost concern. While machine learning based fault management solutions help attain the desired reliability, they require labeled cloud metrics data for training and evaluation. Furthermore, high dynamicity of cloud environments brings forth emerging data distributions, which necessitate frequent labeling of data for model adaptation. We propose a test suite-based active learning framework for automated labeling of cloud metrics data with the corresponding cloud system state while accounting for emerging fault patterns and data or concept drifts. We have implemented our solution on a cloud testbed and introduced various emerging data distribution scenarios to evaluate the proposed framework’s labeling efficacy over known and emerging data distributions. According to our results, the proposed framework achieves a 41% higher weighted Fl-score and a 34% higher average AUC score than a system without any adaptation for emerging data distributions.