Factors Affecting Compilable State at Each Keystroke in CS1

Abstract

In this paper, we analyze keystroke log data from two introductory programming courses from two distinct contexts to investigate the proportion of events that compile, how this relates to contextual factors, the progression of programs, and academic outcomes. We find that, as students write their programs, frequency of compile and run events increases as does the proportion of events that compile. We also find a spike in the number of compile and run events as a program nears completion, that the proportion of events that compile varies by assignment, length of program, and programming context, that real-time IDE error diagnostics lead to higher proportion of events that are in compilable state, and that a student’s awareness of their compilable state is correlated with exam score while the amount of time they spend in an uncompilable state is not. Among the practical implications of our work are the fact that researchers cannot rely on frequency of compilation remaining constant through an assignment and a call to researchers and practitioners to design pedagogies that enhance student awareness of their compilable state.