CLG: Automated checklist generation for improved pull request quality

IF 7.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Expert Systems with Applications Pub Date : 2025-03-14 DOI:10.1016/j.eswa.2025.127178

Shuotong Bai, Chenkun Meng, Guodong Li, Huaxiao Liu, Lei Liu

{"title":"CLG: Automated checklist generation for improved pull request quality","authors":"Shuotong Bai, Chenkun Meng, Guodong Li, Huaxiao Liu, Lei Liu","doi":"10.1016/j.eswa.2025.127178","DOIUrl":null,"url":null,"abstract":"<div><div>The Pull-Based development model, widely embraced in open-source software (OSS), leverages closer global collaboration. However, the increasing number of contributors introduces challenges, particularly in PR quality. Mature repositories address this by using checklists in Pull Request Templates (PRTs). Despite their benefits, a mere 14.15% of popular GitHub repositories implement such checklists. To address this gap, we propose CLG, a Check-List Generation approach, utilizing techniques with a multi-label classifier and summary generation to automatically generate checklists from contributing guidelines. Evaluation results demonstrate CLG’s superiority in each sub-task. As for categorize paragraphs of contributing guidelines, CLG improves 9.81%, 8.44%, 11.84%, and 10.17% across metrics of Accuracy, Precision, Recall, and F1-score. In the task of note generation, CLG improves 16.33%, 15.80%, and 6.43% in terms of ROUGE metrics. To investigate our generated checklists from actual perspectives, we submit our results to the open-source community. The results show that our approach can help the majority of repository maintainers and the open-source community in managing and submitting their received PRs.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"277 ","pages":"Article 127178"},"PeriodicalIF":7.5000,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Expert Systems with Applications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0957417425008000","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

The Pull-Based development model, widely embraced in open-source software (OSS), leverages closer global collaboration. However, the increasing number of contributors introduces challenges, particularly in PR quality. Mature repositories address this by using checklists in Pull Request Templates (PRTs). Despite their benefits, a mere 14.15% of popular GitHub repositories implement such checklists. To address this gap, we propose CLG, a Check-List Generation approach, utilizing techniques with a multi-label classifier and summary generation to automatically generate checklists from contributing guidelines. Evaluation results demonstrate CLG’s superiority in each sub-task. As for categorize paragraphs of contributing guidelines, CLG improves 9.81%, 8.44%, 11.84%, and 10.17% across metrics of Accuracy, Precision, Recall, and F1-score. In the task of note generation, CLG improves 16.33%, 15.80%, and 6.43% in terms of ROUGE metrics. To investigate our generated checklists from actual perspectives, we submit our results to the open-source community. The results show that our approach can help the majority of repository maintainers and the open-source community in managing and submitting their received PRs.

Abstract Image

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Expert Systems with Applications 工程技术-工程：电子与电气

CiteScore

13.80

自引率

10.60%

发文量

2045

审稿时长

8.7 months

期刊介绍： Expert Systems With Applications is an international journal dedicated to the exchange of information on expert and intelligent systems used globally in industry, government, and universities. The journal emphasizes original papers covering the design, development, testing, implementation, and management of these systems, offering practical guidelines. It spans various sectors such as finance, engineering, marketing, law, project management, information management, medicine, and more. The journal also welcomes papers on multi-agent systems, knowledge management, neural networks, knowledge discovery, data mining, and other related areas, excluding applications to military/defense systems.