Change-Based Test Selection in the Presence of Developer Tests

Abstract

Regression test selection (i.e., selecting a subset of a given regression test suite) is a problem that has been studied intensely over the last decade. However, with the increasing popularity of developer tests as the driver of the test process, more fine-grained solutions are in order. In this paper we investigate how method-level changes in the base-code can serve as a reliable indicator for identifying which tests need to be rerun. We validate the approach on two cases - PMD and Cruise Control - using mutation testing as a means to compare the selected subset against a "retest all" approach. Our results show that we are able to reach a sizable reduction of the complete test suite, yet with a comparable number of mutants killed by the reduced test suite.