Estimating the number of residual defects [in software]

Abstract

The number of residual defects is one of the most important factors that allows one to decide if a piece of software is ready to be released. In theory, one can find all the defects and count them. However, it is impossible to find all the defects within a reasonable amount of time. Estimating the defect density can become difficult for high-reliability software, since the remaining defects can be extremely hard to test for. One possible way is to apply the exponential software reliablility growth model (SRGM), and thus estimate the total number of defects present at the beginning of testing. In this paper, we show the problems with this approach and present a new approach based on software test coverage. Test coverage directly measures the thoroughness of testing, avoiding the problem of variations of test effectiveness. We apply this model to actual test data in order to project the residual number of defects. This method results in estimates that are more stable than the existing methods. The method is also easier to understand, and the convergence to the estimate can be observed visually.