Investigating the effects of PuppyCodeReview, an AI-based code review system, on students’ cognitive load

IF 2.4 4区计算机科学 Q2 COMPUTER SCIENCE, SOFTWARE ENGINEERING

SoftwareX Pub Date : 2025-07-30 DOI:10.1016/j.softx.2025.102283

Chun-Hsiung Tseng, Hao-Chiang Koong Lin, Andrew Chih-Wei Huang, Jia-Rou Lin

{"title":"Investigating the effects of PuppyCodeReview, an AI-based code review system, on students’ cognitive load","authors":"Chun-Hsiung Tseng, Hao-Chiang Koong Lin, Andrew Chih-Wei Huang, Jia-Rou Lin","doi":"10.1016/j.softx.2025.102283","DOIUrl":null,"url":null,"abstract":"<div><div>This study presents <em>PuppyCodeReview</em>, an AI-based system designed to support code review activities in programming education. While peer code review is increasingly emphasized in computer science curricula, its effectiveness in university classrooms remains uncertain, especially among novice programmers. <em>PuppyCodeReview</em> aims to address this gap by automating the review process and providing structured feedback on design, functionality, complexity, and common code smells. To evaluate the system’s impact, a four-week experiment was conducted in a data structures course at a university in Taiwan. Students were divided into control and experimental groups; the control group used a standard web interface, while the experimental group utilized the AI-assisted <em>PuppyCodeReview</em> system for the same tasks. Cognitive load was measured before and after the intervention. Results indicated that although cognitive load increased in both groups, the increase was significantly smaller for students using <em>PuppyCodeReview</em>, suggesting the system reduced mental effort associated with code review. These findings highlight the potential of AI-assisted tools in making peer review more accessible and pedagogically effective for novice programmers.</div></div>","PeriodicalId":21905,"journal":{"name":"SoftwareX","volume":"31 ","pages":"Article 102283"},"PeriodicalIF":2.4000,"publicationDate":"2025-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"SoftwareX","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S235271102500250X","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

This study presents PuppyCodeReview, an AI-based system designed to support code review activities in programming education. While peer code review is increasingly emphasized in computer science curricula, its effectiveness in university classrooms remains uncertain, especially among novice programmers. PuppyCodeReview aims to address this gap by automating the review process and providing structured feedback on design, functionality, complexity, and common code smells. To evaluate the system’s impact, a four-week experiment was conducted in a data structures course at a university in Taiwan. Students were divided into control and experimental groups; the control group used a standard web interface, while the experimental group utilized the AI-assisted PuppyCodeReview system for the same tasks. Cognitive load was measured before and after the intervention. Results indicated that although cognitive load increased in both groups, the increase was significantly smaller for students using PuppyCodeReview, suggesting the system reduced mental effort associated with code review. These findings highlight the potential of AI-assisted tools in making peer review more accessible and pedagogically effective for novice programmers.

查看原文本刊更多论文

研究基于人工智能的代码审查系统PuppyCodeReview对学生认知负荷的影响

本研究提出了PuppyCodeReview，一个基于人工智能的系统，旨在支持编程教育中的代码审查活动。虽然同行代码审查在计算机科学课程中越来越受到重视，但其在大学课堂上的有效性仍然不确定，尤其是在新手程序员中。PuppyCodeReview旨在通过自动化审查过程并提供有关设计、功能、复杂性和常见代码气味的结构化反馈来解决这一差距。为了评估该系统的影响，我们在台湾一所大学的数据结构课程中进行了为期四周的实验。将学生分为对照组和实验组；对照组使用标准的web界面，实验组使用人工智能辅助的PuppyCodeReview系统完成相同的任务。在干预前后测量认知负荷。结果表明，尽管两组学生的认知负荷都有所增加，但使用PuppyCodeReview的学生的认知负荷增加幅度要小得多，这表明该系统减少了与代码审查相关的脑力劳动。这些发现突出了人工智能辅助工具在使新手程序员更容易获得同行评审和教学效率方面的潜力。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

SoftwareX COMPUTER SCIENCE, SOFTWARE ENGINEERING-

CiteScore

5.50

自引率

2.90%

发文量

184

审稿时长

9 weeks

期刊介绍： SoftwareX aims to acknowledge the impact of software on today''s research practice, and on new scientific discoveries in almost all research domains. SoftwareX also aims to stress the importance of the software developers who are, in part, responsible for this impact. To this end, SoftwareX aims to support publication of research software in such a way that: The software is given a stamp of scientific relevance, and provided with a peer-reviewed recognition of scientific impact; The software developers are given the credits they deserve; The software is citable, allowing traditional metrics of scientific excellence to apply; The academic career paths of software developers are supported rather than hindered; The software is publicly available for inspection, validation, and re-use. Above all, SoftwareX aims to inform researchers about software applications, tools and libraries with a (proven) potential to impact the process of scientific discovery in various domains. The journal is multidisciplinary and accepts submissions from within and across subject domains such as those represented within the broad thematic areas below: Mathematical and Physical Sciences; Environmental Sciences; Medical and Biological Sciences; Humanities, Arts and Social Sciences. Originating from these broad thematic areas, the journal also welcomes submissions of software that works in cross cutting thematic areas, such as citizen science, cybersecurity, digital economy, energy, global resource stewardship, health and wellbeing, etcetera. SoftwareX specifically aims to accept submissions representing domain-independent software that may impact more than one research domain.