SPICA: A Methodology for Reviewing and Analysing Fault Localisation Techniques

Spectrum-Based Fault Localisation (SBFL) is a well-known technique to find faulty statements in a program. To date, various techniques aiming to improve SBFL from different aspects have been proposed, following their own theories and assumptions. Therefore, it is challenging to make a fair assessment of their rationale and practicability. In this paper, we propose the SPectra Illustration for Comprehensive Analysis (SPICA), a methodology for reviewing and analysing existing SBFL works using spectrum visualisation. Specifically, taking as input a specific SBFL technique (e.g., a suspiciousness metric), SPICA illustrates the relevant artefacts within the spectrum space and then analyse the visualised spectra distribution following the steps of 1) examining the Geometric Characteristics (GCs) and 2) knowledge mining. In this way, we can do overall reviews for various SBFL techniques and get analysis results. As examples, we use SPICA to analyse five representative SBFL techniques, which provide fundamental theories or experimental results. Finally, we provide an overall assessment of the rationale for each technique, attached with suggestions that could be useful for future validation and extension.