Student Engagement Recognition Using Multimodal Fusion Analytical Technology

Abstract

In the application of education, it is very necessary to evaluate student engagement, which is the premise of ensuring teaching quality and implementing teaching intervention. With the development of Internet of things and storage technology, multimodality data acquisition becomes more and more convenient. Considerable research has been devoted to utilizing multimodality data for better understanding student engagement. However, a core research issue has not yet been adequately addressed. Once a set of modalities has been identified, how do we fuse these modalities in an optimal way to perform student engagement analysis? In this paper, we propose a feature fusion framework based on learning process. There are two key steps in the framework, one is the extraction of unequal interval features, and the other is synchronous and asynchronous timing fusion. In addition, we carried out experimental research in real educational scenes, which using image data, log data and text data in online learning to detective different student engagement patterns. The experimental results show the effectiveness and applicability of the framework.