Predicting Effectiveness of Automatic Testing Tools

Abstract

Automatic white-box test generation is a challenging problem. Many existing tools rely on complex code analyses and heuristics. As a result, structural features of an input program may impact tool effectiveness in ways that tool users and designers may not expect or understand. We develop a technique that uses structural program metrics to predict the test coverage achieved by three automatic test generation tools. We use coverage and structural metrics extracted from 11 software projects to train several decision tree classifiers. Our experiments show that these classifiers can predict high or low coverage with success rates of 82% to 94%.