Rethinking time-on-task estimation with outlier detection accounting for individual, time, and task differences

Time-on-task estimation, measured as the duration between two consecutive clicks using student log-files data, has been one of the most frequently used metrics in learning analytics research. However, the process of handling outliers (i.e., excessively long durations) in time-on-task estimation is under-explored and often not explicitly reported in many studies. One common approach to handle outliers in time-to-task estimation is to 'trim' all durations using a cut-off threshold, such as 60 or 30 minutes. This paper challenges this existing approach by demonstrating that the treatment of outliers in an educational context should be individual-specific, time-specific, and task-specific. In other words, what can be considered as outliers in time-on-task depends on the learning pattern of each student, the stages during the learning process, and the nature of the task involved. The analysis showed that predictive models using time-on-task estimation accounting for individual, time, and task differences could explain 3--4% more variances in academic performance than models using an outlier trimming approach. As an implication, this study provides a theoretically grounded and replicable outlier detection approach for future learning analytics research when using time-on-task estimation.