Evaluation of The Cost of Alternate Mutation Strategies

Abstract

Amongst the various testing strategies, mutation testing has been empirically found to be the most effective in detecting faults. However, mutation often imposes unacceptable demands on computing and human resources due to the large number of mutants that need to be compiled and executed on one or more test cases. In addition, the tester needs to examine many mutants and analyze these for possible equivalence with the program under test. For these reasons, mutation is generally regarded by the praeticing test engineer as too expensive to use. As one significant component of the cost of mutation is the execution of mutants against test cases, we believe that the cost can be reduced dramatically by reducing the number of mutants that need to be examined. We report the results from a case study designed to investigate two alternatives to reduce the cost of mutation. The alternatives considered are: (1) randomly selected 2% mutation and (2) constrained mutation. We provide experimental data indicating that both alternatives lead to test sets that distinguish a significant number of all mutants and provide high all-uses coverage.