SLB-Mamba: A vision Mamba for closed and open-set student learning behavior detection

Abstract

By effectively analyzing the learning behaviors of smart classroom students in the classroom, the interaction between teaching and learning can be significantly improved, thereby enhancing the quality of education. However, current traditional analysis of students’ classroom behavior mainly focuses on closed-set behavior detection in a single scenario. In the face of complex and open real classroom environments, obtaining meaningful behavior representations in small and densely populated complex scenarios while achieving good performance in both closed and open environments remains a major challenge. To address these challenges, this study introduces a new method called SLB-Mamba to detect students’ learning behaviors in both closed-set and open-set scenarios. The SLB-Mamba network offers high computational efficiency and flexibility in deployment and practical applications. Firstly, an Attention calculation method Reward-Weighted Attention (RWA) based on the concept of benefit value was designed to enhance the feature extraction ability of the backbone network. Additionally, the Vision State Space Feature Pyramid Network (VSSFPN) structure built through State Space Model (SSM) can effectively integrate cross-scale features. The effectiveness of SLB-Mamba has been validated through rigorous testing and evaluation on real classroom data of smart classrooms, and it has been compared with state-of-the-art (SOTA) methods. The experimental results show that SLB-Mamba achieved mean Average Precision (mAP) scores of 93.79% and 92.2% on the SLB-K12 and SCSB datasets, respectively, with the Absolute Open-Set Error (A-OSE) values of 163 and 289. These findings highlight the significant advantages of the proposed method in improving detection accuracy and efficiency in both closed-set and open-set scenarios, thereby extending the applicability of the educational assessment framework. The source code of this study is publicly available at https://github.com/CCNUZFW/SLB-Mamba.