Leveraging Change Intents for Characterizing and Identifying Large-Review-Effort Changes

Abstract

Code changes to software occur due to various reasons such as bug fixing, new feature addition, and code refactoring. In most existing studies, the intent of the change is rarely leveraged to provide more specific, context aware analysis. In this paper, we present the first study to leverage change intent to characterize and identify Large-Review-Effort (LRE) changes regarding review effort---changes with large review effort. Specifically, we first propose a feedback-driven and heuristics-based approach to obtain change intents. We then characterize the changes regarding review effort by using various features extracted from change metadata and the change intents. We further explore the feasibility of automatically classifying LRE changes. We conduct our study on a large-scale project from Microsoft and three large-scale open source projects, i.e., Qt, Android, and OpenStack. Our results show that, (i) code changes with some intents are more likely to be LRE changes, (ii) machine learning based prediction models can efficiently help identify LRE changes, and (iii) prediction models built for code changes with some intents achieve better performance than prediction models without considering the change intent, the improvement in AUC can be up to 19 percentage points and is 7.4 percentage points on average. The tool developed in this study has already been used in Microsoft to provide the review effort and intent information of changes for reviewers to accelerate the review process.