Mining Code Submissions to Elucidate Disengagement in a Computer Science MOOC

Abstract

Despite the growing prevalence of Massive Open Online Courses (MOOCs) in the last decade, using them effectively is still challenging. Particularly, when MOOCs involve teaching programming, learners often struggle with writing code without sufficient support, which may increase frustration, attrition, and eventually dropout. In this study, we assess the pedagogical design of a fresh introductory computer science MOOC. Keeping in mind MOOC “end-user” instructors, our analyses are based merely on features easily accessible from code submissions, and methods that are relatively simple to apply and interpret. Using visual data mining we discover common patterns of behavior, provide insights on content that may require reevaluation and detect critical points of attrition in the course timeline. Additionally, we extract students’ code submission profiles that reflect various aspects of engagement and performance. Consequently, we predict disengagement towards programming using classic machine learning methods. To the best of our knowledge, our definition for attrition in terms of disengagement towards programming is novel as it suits the unique active hands-on nature of programming. To our perception, the results emphasize that more attention and further research should be aimed at the pedagogical design of hands-on experience, such as programming, in online learning systems.