Continuous test suite augmentation in software product lines

Software Product Line (SPL) engineering offers several advantages in the development of families of software products. There is still a need, however, to generate test cases for individual products in product lines more efficiently. In this paper we propose an approach, CONTESA, for generating test cases for SPLs using test suite augmentation. Instead of generating test cases for products independently, our approach generates new test cases for products in an order that allows it to build on test cases created for products tested earlier. In this work, we use a genetic algorithm to generate test cases, targeting branches not yet covered in each product, although other algorithms and coverage criteria could be utilized. We have evaluated CONTESA on two non-trivial SPLs, and have shown that CONTESA is more efficient and effective than an approach that generates test cases for products independently. A further evaluation shows that CONTESA is more effective at achieving coverage, and reveals as many faults as an existing feature-based testing approach.