Theoretical, Weak and Strong Accuracy Graphs of Spectrum-Based Fault Localization Formulas

Driven by the need to know which spectrum-based fault localization techniques are more effective in locating faults, many studies have sought to compare the accuracy of different formulas used in these techniques, resulting in findings of both theoretical and empirical accuracy relations of these formulas. Theoretical accuracy relations are independent of the specific programs and other settings involved, but limited by underlying assumptions and manual work in proofs. An accuracy graph can be constructed to holistically represent the proved relations. On the other hand, empirical studies are free of specific theoretical assumptions and can be highly automatable and scalable. A recent study has developed a systematic methodology based on statistical tests to reveal consistent and statistically sound empirical accuracy relations. That work has demonstrated the merits of empirical accuracy graphs in revealing relations that can be hard to prove. In this paper, we propose to use a stronger criterion for comparing formulas, describe an exploratory experiment to construct accuracy graphs based on the criterion, and report interesting relations found from the resulting accuracy graphs.