Validating the defect detection performance advantage of group designs for software reviews: report of a laboratory experiment using program code

Abstract

It is widely accepted that software development technical reviews (SDTRs) are a useful technique for finding defects in software products. Recent debates centre around the need for review meetings (Porter and Votta 1994, Porter et al 1995, McCarthy et al 1996, Lanubile and Visaggio 1996). This paper presents the findings of an experiment that was conducted to investigate the performance advantage of interacting groups over average individuals and artificial (nominal) groups. We found that interacting groups outperform the average individuals and nominal groups. The source of performance advantage of interacting groups is not in finding defects, but rather in discriminating between true defects and false positives. The practical implication for this research is that nominal groups constitute an alternative review design in situations where individuals discover a low level of false positives.