Reflective Debugging in Spinoza V3.0

In this paper we present an online IDE (Spinoza 3.0) for teaching Python programming in which the students are (sometimes) required to verbally reflect on their error messages and unit test failures before being allowed to modify their code. This system was designed to be used in large synchronous in-person, remote, or hybrid classes for either in-class problem solving or out-of-class homework problems. For each student and problem, the system makes a random choice about whether to require reflection on all debugging steps. If the student/problem pair required reflection, then after each time the student ran the program and received feedback as an error message or a set of unit test results, they were required to type in a description of the bug and a plan for how to modify the program to eliminate the bug. The main result is that the number of debugging steps to reach a correct solution was statistically significantly less for problems where the students were required to reflect on each debugging step. We suggest that future developers of pedagogical IDEs consider adding features which require students to reflect frequently during the debugging process.