Test case prioritization and mutation testing

Abstract

In this issue, we are pleased to present two papers on test case prioritization and mutation testing, respectively.

The first paper, ‘Semantic-aware two-phase test case prioritization for continuous integration’ by Yingling Li, Ziao Wang, Junjie Wang, Jie Chen, Rui Mou, and Guibing Li, presents the SatTCP framework to conduct precise prioritization with low time overhead, in order to improve the cost effectiveness of typical continuous integration (CI) testing with frequent code submissions. In SatTCP, coarse-grained filtering based on information retrieval (IR) techniques roughly sorts test cases and selects a certain number of tests for the subsequent prioritization; then fine-grained prioritization based on pretrained Siamese network conducts precise prioritization of initially ranked test sets. The evaluation results show that SatTCP outperforms all the baselines under comparison, and achieves the lowest test costs. (Recommended by Yves Le Traon).

The second paper, ‘Mutation testing optimisations using the Clang front-end’ by Sten Vercammen, Serge Demeyer, Markus Borg, Niklas Pettersson, and Görel Hedin, presents an investigation to which extent the Clang front-end and its state-of-the-art program analysis facilities allow to implement existing strategies for mutation optimization within the C language family. The authors develop a proof-of-concept tool used to collect detailed measurements for each mutation phase. The authors conduct evaluation of the proof-of-concept tool on four open-source C++ libraries and one industrial component. The evaluation results show that the ‘Generate Mutants’ and ‘Detect (Un)Reachable Mutants’ steps are for all practical purposes negligible; the ‘Compile Mutants’ step takes a significant amount of time and the compilation of the invalid and unreachable mutants is considerable; the ‘Execute Mutants’ step is the other dominant factor. (Recommended by Mike Papadakis).

We hope that these papers will inspire further research in related directions.