Similarity-based prioritization in software product-line testing

Abstract

Exhaustively testing every product of a software product line (SPL) is a difficult task due to the combinatorial explosion of the number of products. Combinatorial interaction testing is a technique to reduce the number of products under test. However, it is typically up-to the tester in which order these products are tested. We propose a similarity-based prioritization to be applied on these products before they are generated. The proposed approach does not guarantee to find more errors than sampling approaches, but it aims at increasing interaction coverage of an SPL under test as fast as possible over time. This is especially beneficial since usually the time budget for testing is limited. We implemented similarity-based prioritization in FeatureIDE and evaluated it by comparing its outcome to the default outcome of three sampling algorithms as well as to random orders. The experiment results indicate that the order with similarity-based prioritization is better than random orders and often better than the default order of existing sampling algorithms.