Integrating Blended Learning Behaviors via Multimodal Fusion for Student Performance Prediction

Abstract

Blended learning enriches students’ experiences across diverse environments while generating multimodal data related to learning activities. However, it presents challenges in the appropriate use of multimodal data to track students’ performance development. Previous models with fixed-length inputs or static fusion mechanisms inadequately model temporal dependencies across behavioral modalities. In this article, we integrate variable-length time series over weeks to forecast performance for the subsequent week. As the main contribution, we propose a two-stage training model that relies on a transformer for temporal attention-based multimodal fusion. We conducted experiments on two real-world datasets, FC2023 and CS2023, derived from hybrid mode courses involving 439 and 199 students, respectively. The results demonstrate that multimodal fusion yields better periodical prediction compared to the unimodal approach. Ultimately, aiming at predicting the week-by-week development of student performance, the proposed model achieves the area under the curve of receiver operating characteristic of 81.02% on FC2023 and 82.65% on CS2023. This approach, which leverages multimodal learning analytics, helps educators track each student’s learning progress more effectively, enabling the timely implementation of instructional interventions and enhancing educational outcomes.