Coverage, Location, Detection, and Measurement

Abstract

Complex engineered systems arise throughout computing, communications, and networking. Many factors, each having a finite number of levels, impact the behaviour of the system either singly or in interaction with one another. Testing or evaluating such a system involves formulating a set of tests, when executed, responses or outcomes from the tests are analyzed. A single round of testing is conducted. To witness the effect of an interaction, some test must cover it, this does not suffice in general to locate the interaction or to measure its effect. When there are few factors or many tests, experimental designs can measure (and hence locate) the interactions. When there are many factors and few tests, can we locate the interaction(s)? Can we efficiently detect them?Combinatorial arrays, locating and detecting arrays, are introduced to address such location and detection in the context of combinatorial testing. Locating and detecting arrays are contrasted with covering arrays and with experimental designs. An application to a 75 factor protocol stack for file transfer is given to demonstrate their practical use. Finally, their place in the literature of combinatorial testing is discussed and some directions are outlined.