Measuring Students’ Self-Regulatory Phases in LMS with Behavior and Real-Time Self Report

Abstract

Research has emphasized that self-regulated learning (SRL) is critically important for learning. However, students have different capabilities of regulating their learning processes and individual needs. To help students improve their SRL capabilities, we need to identify students’ current behaviors. Specifically, we applied instructional design to create visible and meaningful markers of student learning at different points in time in LMS logs. We adopted knowledge engineering to develop a framework of proximal indicators representing SRL phases and evaluated them in a quasi-experiment in two different learning activities. A comparison of two sources of collected students’ SRL data, self-reported and trace data, revealed a relatively high agreement between our classifications (weighted kappa, κ = .74 and κ = .68). However, our indicators did not always discriminate adjacent SRL phases, particularly for enactment and adapting phases, compared with students’ real-time self-reported behaviors. Our behavioral indicators also were comparably successful at classifying SRL phases for different self-regulatory engagement levels. This study demonstrated how the triangulation of various sources of students’ self-regulatory data could help to unravel the complex nature of metacognitive processes.