iTest: Using coverage measurements to improve test efficiency

Abstract

Many testing tasks in practice are still done manually. Which test cases are developed, automated and selected for execution is done by experience and instinct, rather than facts and data from the current system. To change this automated tool support requires lots of different data from the system under test. One very important information is what parts of the system are reached by a given test. This coverage information can be used to select, prioritize, or remove tests for the tested system revision. In this paper, we discuss our work on recording code coverage for individual system level tests. The discussed approach has shown to work in industry and is already in use. Nonetheless, we still have several open questions that require answers to improve coverage recordings. We highlight the lessons learned from our ongoing work and discuss the open questions to encourage further research on these problems.