Boosting mutation-based fault localization by effectively generating Higher-Order Mutants

Abstract

Context:

Fault Localization (FL) is an important and tedious phase of software debugging. Among various FL techniques, Mutation-Based Fault Localization (MBFL) demonstrates promising FL accuracy utilizing impact information of statements provided by First-Order-Mutants (FOMs). Despite its success in Single-Fault Scenarios (SFSs), it fails to achieve satisfactory performance in Multiple-Fault Scenarios (MFSs). Higher-Order-Mutants (HOMs) provide a solution for MFSs. However, existing work on HOM generation is inadequate and ignores the correlation among faults in MFSs.

Objective:

In this article, we systematically analyze three relationships among single-faults in MFSs, and further propose three HOM generation methods (i.e., SFClu, SFDis, and SFDen) to simulate different multiple-faults and improve the effectiveness of MBFL in MFSs.

Method:

We investigate the multiple-fault composition on real-world 393 faulty programs from Defects4J, and then apply our methods to generate HOMs for FL. Specifically, SFClu focuses on generating appropriate HOMs for Multi-Single-Source Fault (MSSF) scenarios, where each single-fault is responsible for different observed failures. SFDis is well-suited for Multi-Coupled-Source Fault (MCSF) scenarios where at least two single-faults can interact with each other, leading to certain failures either being observable or masked. SFDen aims to generate suitable HOMs for Single-Coupled-Source Fault (SCSF) scenarios with multiple single-faults that occur within a statement.

Results:

(1) The proportion of MFSs is as high as 63.10% in real-world programs, with MSSF, MCSF , and SCSF scenarios accounting for 35.08%, 53.23%, and 11.69%, respectively. (2) Compared to the best-performing mutant generation method Neural-MBFL, SFClu, SFDis, and SFDen can improve the FL performance for MBFL by 36.78%, 49.80%, and 16.36% in $Top$-1, respectively, outperforming eight established SBFL and MBFL techniques. (3) SFClu, SFDis, and SFDen are more suitable for MSSF, MCSF, and SCSF scenarios, respectively, which aligns with their design intend. (4) Their combination further enhances FL accuracy, achieving up to 85 faults successfully localized and an average improvement of 29.54% in $Top$-1. Finally, extensive evaluations on SIR with artificial faults and Codeflaws containing student programs demonstrate the generalization of SFClu, the applicability of SFDis on real faults, and the suitability of SFDen for student programs.

Conclusion:

Empirical studies have confirmed the prevalence of MFSs, highlighting the significance of multi-fault localization. Moreover, our proposed three HOM generation methods can further enhance the performance of MBFL with HOMs in MFSs, showing their effectiveness and applicability.