Relating counterexamples to test cases in CTL model checking specifications

Abstract

Counterexamples produced by model checkers are frequently exploited for the purpose of testing. Counterexamples and test cases are generally treated as essentially the same thing, while in fact they can differ significantly. For example, it might take more than one test case to "cover" a given counterexample, because not all property violations can be illustrated with linear counterexamples. This paper presents a formal relationship between counterexamples and test cases in the context of the Computation Tree Logic (CTL), the logic of the popular model checker SMV. Given a test requirement as a CTL formula, we define what it means for a set of test cases to cover a counterexample associated with that requirement. This result can not only be used in the generation of a test set that satisfies a given test coverage criterion, but also in the determination of whether an extant test set satisfies the criterion. Our results can guide the production of counterexamples in model checkers explicitly intended to support testing.