Evaluating the Fault-Focused Clustering Performance of Distance Metrics in Parallel Fault Localization: From an Omniscient Perspective

Abstract

To locate multiple bugs in parallel, one common practice is to generate fault-focused clusters where failed test cases that are likely caused by the same bug are grouped together. With respect to the fault-focused clustering performance, a critical impact factor is the distance metric used to measure the similarity between two rankings. This paper proposes a method to evaluate the fault-focused clustering performance of distance metrics from an omniscient perspective where the fault-focused information for each failed test case is already given. Case studies are conducted using the proposed method to evaluate Jaccard and Kendall tau distance on three programs with multiple bugs. The findings seem to challenge previous perceptions regarding the performance of these two distance metrics in generating fault-focused clusters.