Using submission log data to investigate novice programmers’ employment of debugging strategies

Abstract

Debugging is a distinct subject in programming that is both comprehensive and challenging for novice programmers. However, instructors have limited opportunities to gain insights into the difficulties students encountered in isolated debugging processes. While qualitative studies have identified debugging strategies that novice programmers use and how they relate to theoretical debugging frameworks, limited larger scale quantitative analyses have been conducted to investigate how students’ debugging behaviors observed in log data align with the identified strategies and how they relate to successful debugging. In this study, we used submission log data to understand how the existing debugging strategies are employed by students in an introductory CS course when solving homework problems. We identified strategies from existing debugging literature that can be observed with trace data and extracted features to reveal how efficient debugging is associated with debugging strategy usage. Our findings both align with and contradict past assumptions from previous studies by suggesting that minor code edition can be a beneficial strategy and that width and depth aggregations of the same debugging behavior can reveal opposite effects on debugging efficiency.