Are They Learning or Guessing? Investigating Trial-and-Error Behavior with Limited Test Attempts

Abstract

Mastery learning and deliberate practice promote personalized learning, allowing the learner to improve through a repetitive and targeted approach. Unfortunately, this pedagogy is challenging to implement in classrooms where everyone is expected to learn at the same pace. Various studies have successfully demonstrated that certain aspects of mastery learning can be integrated into the curriculum. We adopt a similar pedagogical strategy and explain our implementation approach with online assessments. Since this is a new pedagogical approach to assessing student learning, we collected data to investigate test-taking behavior and evaluated potential learning gains in this new test format. As part of this endeavor, we developed a model to detect trial-and-error sequences in test attempts. Our results point to a small percentage of guessing behavior, which is encouraging evidence supporting this test approach is a viable way to implement mastery learning in our curriculum.