Covering arrays: Evaluating coverage and diversity in the presence of disallowed combinations

Test engineers are often faced with the challenge of selecting test cases that maximize the chance of discovering faults while working with a limited budget. Combinatorial testing is an effective test case selection strategy to address this challenge. The basic idea is to select test cases that ensure that all possible combinations of settings from two (or more) inputs are accounted for, regardless of which subset of two (or more) inputs are selected. Currently, combinatorial testing usually implies a covering array as the underlying mathematical construct. Yet, despite their demonstrated utility, practitioners sometimes encounter challenges that impede their use. For example, given a covering array with constraints on allowed combinations of settings for some subset of inputs, it is often unclear how to assess the coverage and diversity [2] properties of the resulting covering array.