Empirical performance analysis of computer-supported code-reviews

Checklist-based code-reviews have been generally accepted as valuable means for software development and management. In order to overcome shortcomings of manual reviewing techniques, such as high costs and lack of systematization, we have already developed and implemented a knowledge-based approach for semi-automation of some steps of individual code-reviews based on checklists. The aim of this paper is to evaluate the performance of our approach for code-reviews. Therefore, two independent groups of reviewers were analyzing the same piece of code (conventional C-programs developed in the automotive industry for gear unit control), where one of the groups deployed the semi-automated approach and the other group used the traditional manual technique. The resulting empirical data were analyzed by means of software metrics and software reliability modeling. Metrics specific to review processes, e.g. average review rate, average preparation rate, etc. were adapted for our knowledge-based methodology in order to capture the particularities of our approach. The results of these empirical investigations are compared with other methods (e.g. statistical, plan-based scenario-based) for reviews' computer support.