Clustering Automation Test Faults

Abstract

Black-box user interface testing has become a powerful and popular approach in automated software testing. Since the increasing number of test cases which need to be run at each iteration leads to more execution faults, the process of analyzing test failures to find the root cause or to triage usually consumes much effort. Hence, there is a need that these errors be clustered into groups based on their root cause to facilitate debugging and maintenance purposes. In this paper, we propose an automated text clustering approach along with a semi-automated version for clustering errors in term of their root causes which can help save a lot of effort in triaging and fixing bugs. Our experiment uses datasets from three different projects, two of which are industrial ones, with more than 300 errors generated in total. The results show that our approach outperforms other existing baseline methods that are utilized widely in classification and clustering field indicating that the strategy may be effective.