Multimodal depression recognition based on gait and rating scale

Abstract

Recently, depression recognition has garnered significant attention. Given its ease of acquisition from a distance, gait-based depression analysis emerges as a valuable tool for assisting in the diagnosis and assessment of depression. However, current research on gait-based depression recognition often uses scale results as labels but neglects the rich semantic information within the scales, which reflects the emotional, lifestyle, and physical states of participants and provides more personalized depression characteristics. To enhance the reliability and accuracy of depression analysis, we propose a text-guided depression recognition method based on gait. Firstly, we utilize silhouette-based modeling for depression recognition to capture relevant gait features. Secondly, we design the GT-CLIP module to leverage text information from scales as an auxiliary branch to guide feature learning within the gait recognition framework, enabling the model to effectively extract corresponding gait features based on these depression-related text information. Then, we devise a text-guided attention mechanism to capture variations across different body parts. In the D-Gait dataset, which includes 92 depressed subjects and 200 normal controls, our proposed text-guided depression recognition model achieves an F1-score of 59.85, outperforming existing state-of-the-art methods.