Automated Quantification of Parkinsonian Facial Expressions via a Transformer-Based Multi-View Multimodal Fusion Framework.

Abstract

Facial hypomimia, a common symptom of Parkinson's disease (PD), is characterized by reduced facial expressiveness due to muscle rigidity. Its clinical assessment, typically based on the Movement Disorder Society-Unified PD Rating Scale, is challenging due to the limited clinician availability, subjectivity, and time consuming procedures. In this paper, we propose a transformer-based multi-view multimodal fusion framework for efficient and objective video-based automated assessment of PD facial expressions. The framework incorporates facial anatomical priors, jointly models global and local features, and progressively fuses complementary information across modalities. It uses a progressive fusion strategy for cross-modal interaction and compensation, together with a spatial-temporal transformer block for anatomically guided landmark modeling of global motion and local subtle dynamics. Our proposed method achieved an accuracy of 68.28% and an acceptable accuracy of 87.75% on the largest clinical dataset (637 videos from 354 PD patients) with four classes. Moreover, testing on an independent dataset from another hospital provides preliminary evidence of the model's cross-center transferability, and the framework was further evaluated on two additional medical facial assessment tasks of stroke facial palsy and facial function. In summary, the proposed framework provides an efficient and reliable solution for the objective assessment of PD facial expressions, with the potential to improve evaluation consistency and support broader clinical applications.