Automatic Depression Recognition Using Multi-scale Facial Behavior Dynamics

Abstract

Depression is currently the most common mental health issue that has negative impacts on a large number of people’s life. Although many recent studies proposed to estimate depression severity from human behaviors, the majority of them failed to consider multi-scale behavioral dynamics, which can be crucial clues for depression recognition. In this paper, we propose a novel system that can encode multi-scale short-term and long-term behavioral dynamics for depression recognition. It first extends Dynamic Image Algorithm to extract multi-scale short-term behavioral dynamic feature time-series at the frame-level using different time-windows. Then, we encode the time-series of frame-level short-term dynamic features of a whole video into a spectral representation, which encodes multi-scale long-term behavioral dynamic features. Finally, we feed this video-level multi-scale dynamic representations to standard ANN for depression severity estimation. The experiment results achieved on AVEC 2017 dataset show that the proposed multi-scale facial dynamic encoding approach can provide accurate depression severity prediction than most existing methods that did not consider such temporal information.