Analyzing novice and competent programmers' problem-solving behaviors using an automated evaluation system

IF 1.5 4区 计算机科学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING
Yung-Ting Chuang, Hsin-Yu Chang
{"title":"Analyzing novice and competent programmers' problem-solving behaviors using an automated evaluation system","authors":"Yung-Ting Chuang,&nbsp;Hsin-Yu Chang","doi":"10.1016/j.scico.2024.103138","DOIUrl":null,"url":null,"abstract":"<div><h3>Background and Context</h3><p>In today's tech-driven world, programming courses are crucial. Yet, teaching programming is challenging, leading to high student failure rates. Understanding student learning patterns is key, but there's a lack of research utilizing tools to automatically collect and analyze interaction data for insights into student performance and behaviors.</p></div><div><h3>Objectives</h3><p>Study aims to compare problem-solving behaviors of novice and competent programmers during coding tests, identifying patterns and exploring relationships with program correctness.</p></div><div><h3>Method</h3><p>We built an online system with programming challenges to collect behavior data from novice and competent programmers. Our system analyzed data using various metrics to explore behavior-program correctness relationships.</p></div><div><h3>Findings</h3><p>Analysis showed distinct problem-solving behavior patterns. Competent programmers had fewer syntax errors, spent less time fixing bugs, and had higher program correctness. Novices made more syntax errors and spent more time fixing coding errors. Both groups used tabs for code structure, but competent programmers introduced unfamiliar variables more often and commented on them afterward. Emphasizing iterative revisions and active engagement enhances problem-solving skills and programming proficiency. Radar charts are effective for identifying improvement areas in teaching programming. The relationship between behavior and program correctness was positively correlated for competent programmers but not novices.</p></div><div><h3>Implications</h3><p>Study findings have implications for programming education. Radar charts help teachers identify course improvement areas. Novices can learn from competent programmers' behavior. Instructors should encourage continuous skill improvement through revisions and engagement. Identified unfamiliar programming aspects offer insights for targeted learning.</p></div>","PeriodicalId":49561,"journal":{"name":"Science of Computer Programming","volume":"237 ","pages":"Article 103138"},"PeriodicalIF":1.5000,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science of Computer Programming","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167642324000613","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0

Abstract

Background and Context

In today's tech-driven world, programming courses are crucial. Yet, teaching programming is challenging, leading to high student failure rates. Understanding student learning patterns is key, but there's a lack of research utilizing tools to automatically collect and analyze interaction data for insights into student performance and behaviors.

Objectives

Study aims to compare problem-solving behaviors of novice and competent programmers during coding tests, identifying patterns and exploring relationships with program correctness.

Method

We built an online system with programming challenges to collect behavior data from novice and competent programmers. Our system analyzed data using various metrics to explore behavior-program correctness relationships.

Findings

Analysis showed distinct problem-solving behavior patterns. Competent programmers had fewer syntax errors, spent less time fixing bugs, and had higher program correctness. Novices made more syntax errors and spent more time fixing coding errors. Both groups used tabs for code structure, but competent programmers introduced unfamiliar variables more often and commented on them afterward. Emphasizing iterative revisions and active engagement enhances problem-solving skills and programming proficiency. Radar charts are effective for identifying improvement areas in teaching programming. The relationship between behavior and program correctness was positively correlated for competent programmers but not novices.

Implications

Study findings have implications for programming education. Radar charts help teachers identify course improvement areas. Novices can learn from competent programmers' behavior. Instructors should encourage continuous skill improvement through revisions and engagement. Identified unfamiliar programming aspects offer insights for targeted learning.

使用自动评估系统分析程序员新手和能手解决问题的行为
背景在当今技术驱动的世界中,编程课程至关重要。然而,编程教学充满挑战,导致学生失败率居高不下。了解学生的学习模式是关键,但目前缺乏利用工具自动收集和分析交互数据以深入了解学生表现和行为的研究。研究旨在比较新手和有能力的程序员在编码测试中解决问题的行为,找出模式并探索与程序正确性之间的关系。我们的系统使用各种指标对数据进行分析,以探索行为与程序正确性之间的关系。有能力的程序员语法错误较少,修复错误的时间较短,程序正确率较高。新手则语法错误较多,修复编码错误的时间较长。两组程序员在代码结构上都使用了选项卡,但有能力的程序员会更多地引入陌生变量,并在事后对其进行注释。强调迭代修改和积极参与能提高解决问题的能力和编程熟练度。雷达图能有效确定编程教学中的改进领域。对于有能力的程序员而言,行为与程序正确性之间呈正相关,而对于新手则不然。雷达图有助于教师确定课程改进领域。新手可以从合格程序员的行为中学习。教师应鼓励学生通过修改和参与不断提高技能。发现的不熟悉的编程方面为有针对性的学习提供了启示。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Science of Computer Programming
Science of Computer Programming 工程技术-计算机:软件工程
CiteScore
3.80
自引率
0.00%
发文量
76
审稿时长
67 days
期刊介绍: Science of Computer Programming is dedicated to the distribution of research results in the areas of software systems development, use and maintenance, including the software aspects of hardware design. The journal has a wide scope ranging from the many facets of methodological foundations to the details of technical issues andthe aspects of industrial practice. The subjects of interest to SCP cover the entire spectrum of methods for the entire life cycle of software systems, including • Requirements, specification, design, validation, verification, coding, testing, maintenance, metrics and renovation of software; • Design, implementation and evaluation of programming languages; • Programming environments, development tools, visualisation and animation; • Management of the development process; • Human factors in software, software for social interaction, software for social computing; • Cyber physical systems, and software for the interaction between the physical and the machine; • Software aspects of infrastructure services, system administration, and network management.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信