Depression Detection Based on Facial Expression, Audio and Gait

Abstract

Depression is a mental illness that endangers patients’ physical and mental health and imposes burdens on family and society. More and more people suffer from depression nowadays, which increases medical pressure. Depression can be diagnosed by patients’ voice, facial expression and gait. The current study mostly bases on one modality or a fusion of two. In this paper, we gathered 234 pieces of gait video, interview audio and video, proposed our pipeline and compared the performance between three single modalities and multi-modal fusion. The facial expression has the best performance, audio comes second, and gait comes last. The fusion of modalities can improve performance. This can provide a basis for the choice of modality in automatic screening or auxiliary diagnosis of depression. We also evaluated our model on public data set AVEC 2013, AVEC 2014 and Emotion-gait, which verifies its validity.