Coverage Criteria for State-Based Testing: A Systematic Review

Abstract

State-based testing (SBT) is known as deriving test cases from state machines and examining the dynamic behaviour of the system. It helps to identify various types of state-based faults within a system under test (SUT). For SBT, test cases are generated from state chart diagrams based on various coverage criteria such as All Transition, Round Trip Path, All Transition Pair, All Transition Pair with length 2, All Transition Pair with length 3, All Transition Pair of length 4 and Full Predicate. This article discuses a number of coverage criteria at the design level to find out various types of state-based faults in SBT. First, the intermediate graph is generated from a state chart diagram using an XML parser. The graph is traversed based on the given coverage criteria to generate a sequence of test cases. Then, mutation testing and sneak-path testing are applied on the generated test cases to check the effectiveness of the generated test suite. These two are common methods for checking the effectiveness of test cases. Mutation testing helps in the number of seeded errors covered whereas sneak-path testing basically helps to examine the unspecified behavior of the system. In round trip path (RTP), it is not possible to cover all paths. All transition is not an adequate level of fault detection with more execution time compared to all transition pair (ATP) with length 4 (LN4). In the discussion, ATP with LN4 is the best among all coverage criteria. SBT can able to detect various state-based faults-incorrect transition, missing transition, missing or incorrect event, missing or incorrect action, extra missing or corrupt state, which are difficult to detect in code-based testing. Most of these state-based faults can be avoided, if the testing is conducted at the early phase of design.