CodeDoctor: multi-category code review comment generation

Abstract

Code review is an effective software quality assurance activity. However, this process is labor-intensive and time-consuming, requiring reviewers to carefully review under various categories (e.g., function, refactoring, documentation, etc) to generate review comments. Several approaches have been proposed for automatic review comment generation, although they can generate review comments, they hardly cover all manual review comments. Because most of these approaches simply utilize the information of submitted code and review comments, not fully modeling the features of code review (i.e., ignoring review category, the association of issue snippets and review comments). In this paper, we propose CodeDoctor, an automatic review comment generator with data augmentation and category-aware encoder-decoder to generate multi-category review comments. It consists of three main phases: (1) Data augmentation phase, which classifies review comments and builds review exemplars (i.e., the pairs of issue snippet and its comment) to augment review data by using a large language model (LLM) with prompt engineering and feedback loops; (2) Encoder phase, which encodes the inputs (i.e., review category, diff code and review exemplar) into semantic and token representations; (3) Decoder phase, which designs a category-focused decoder to capture the most relevant information of given category for multi-category review comment generation. Evaluations with five commonly-used and state-of-the-art baselines on two datasets show that CodeDoctor outperforms all baselines, with 1770% higher average BLEU-4, 111% higher average ROUGE-L and 49% higher average F1 than the best baseline. Furthermore, a human evaluation also confirms the significant potential of applying CodeDoctor in practical usage. Our approach can relieve the burden of reviewers by automatically generating multi-category review comments, and helps developers better detect code issues as early as possible, thereby facilitating software development.