MI3S: A multimodal large language model assisted quality assessment framework for AI-generated talking heads

Although current speech-driven technologies enable the rapid generation of AI-generated talking heads (AGTHs), human supervision remains necessary to ensure the quality of the output. However, manual evaluation becomes increasingly impractical for large-scale AGTH production due to its time-consuming and labor-intensive nature. To overcome this limitation, we propose a novel objective quality assessment framework, MI3S, which employs a Multimodal Large Language Model (MLLM) to evaluate AGTHs across four key dimensions: Image quality, Image aesthetics, Identity consistency, and Sound-lip synchronization. To capture temporal dynamics more effectively, we introduce a variable-length video memory filter (VVMF), inspired by principles of human visual cognition. The MI3S framework supports both zero-shot inference and supervised learning paradigms. On the THQA dataset comprising 800 AGTHs, MI3S achieves a prediction-human perceptual correlation coefficient of 0.7946, which exceeds that of existing quality assessment methods by 3.4%, thereby offering an efficient, robust, and objective solution for evaluating AGTH quality.