CFGMamba: Cross frame group Mamba for video-based depression recognition

Abstract

Depression recognition is a significant research topic in the field of affective computing, which has important value for promoting clinical diagnosis and screening of depression. Video-based depression recognition methods utilize Convolutional Neural Networks (CNNs) or Transformers to capture relevant visual features and achieve promising performance. However, the limited receptive field of CNNs, the high computational resource consumption of Transformer long sequence modeling, and the high dimensionality of video data are key issues to be addressed. Considering these factors, this work introduces the State Space Model (SSM) for depression recognition and proposes a Cross Frame Group Mamba (CFGMamba) framework. CFGMamba alleviates the limitations of CNNs through global receptive fields and can effectively model long-range sequences with linear complexity. Technically, CFGMamba models cross-frame grouping of video data, dividing video frames into several distinct groups at time intervals and then performing bidirectional scanning for each group in the spatial–temporal dimension. This cross-frame grouping strategy efficiently captures richer emotional features while minimizing computational overhead. Meanwhile, CFGMamba incorporates a multi-stage downsampling approach, where multiple CFGMamba blocks are stacked at each stage to progressively capture multi-scale spatial–temporal emotional features from shallow to deep layers. Experimental results on the AVEC 2013 and AVEC 2014 datasets indicate that CFGMamba achieves competitive performance, with MAE/RMSE of 6.01/7.59 and 5.96/7.52, respectively. And the F1-score/AUC is 0.75/0.78 on the EmoReact dataset.