Vision based Physiological and Emotional Signal Analysis with Application to Mental Disorder Diagnosis

Face images and videos contain rich visual biometric signals from apparent signals like attribute and identity characteristics to subtle signals corresponding to physiological and emotional states. Benefit from the great success of deep learning methods, tremendous progress has been made on apparent visual signals analysis. However, subtle signal analysis still faces big challenges: indistinguishable pattern, low PSNR, and transient duration. Attempts to resolve these challenges usually rely on engineering designs to extract and enhance the subtle signals. Our recent work aims to improve the robustness of physiological and emotional signal analysis via signal disentanglement, context modeling, and semi-supervised learning. Since people with mental disorders is likely to demonstrate subtle visual signals, we also propose to fuse individual face visual signals to perform mental disorder diagnosis like AD apathy and anxiety prediction.