Semantic Mutation Testing

Mutation testing is a powerful and flexible test technique. Traditional mutation testing makes a small change to the syntax of a description (usually a program) in order to create a mutant. A test set is considered to be good if it distinguishes between the original description and all of the (functionally nonequivalent) mutants. These mutants can be seen as representing potential small slips and thus mutation testing aims to produce a test set that is good at finding such slips. It has also been argued that a test set that finds such small changes is likely to find larger changes. This paper introduces a new approach to mutation testing, called semantic mutation testing. Rather than mutate the description, semantic mutation testing mutates the semantics of the language in which the description is written. The mutations of the semantics of the language represent possible misunderstandings of the description language and thus capture a different class of faults. Since the likely misunderstandings are highly context dependent, this context should be used to determine which semantic mutants should be produced. The approach is illustrated through examples with state charts and C code. In addition, a semantic mutation testing tool for C is proposed.