Reduce Test Cost by Reusing Test Oracles through Combinatorial Join

Abstract

Methods to generate combinatorial test suites have been extensively studied in the combinatorial interaction testing (CIT) community, but the creation of test oracles for the test cases remains a challenging and expensive task because they are created manually. In this paper, we propose a novel technique to "join" multiple test suites into one while satisfying the required combinatorial coverage of the system under test. This technique allows test oracles designed and created in earlier testing phases to be reused in later ones. Our study results indicate that the technique can reduce the total test cost to design and execute a test suite by more than 55% compared to the conventional testing approach.