A multi-granularity facial aesthetic evaluation model based on image-text modality

Abstract

Facial Beauty Prediction (FBP) is an emerging research direction at the intersection of artificial intelligence and aesthetics, which has attracted increasing attention in recent years. However, most existing methods rely solely on unimodal data and fail to comprehensively capture the multi-dimensional information of facial aesthetics. To address this challenge, we propose a multigranularity facial aesthetic evaluation model based on image-text modality (ITM-MGFA). By incorporating multi-granularity cognitive theory into the FBP task, the model effectively integrates both coarse-grained and fine-grained aesthetic features extracted from the CLIP encoder through a multigranularity representation module, a task-oriented dynamic alignment module, and a hierarchical interaction optimization module. This facilitates deep cross-modal interaction and fusion, significantly enhancing the model’s capability to model complex aesthetic attributes. Experimental results demonstrate that ITM-MGFA, leveraging the fusion of cross-modal information, achieves higher accuracy in facial aesthetic assessment task compared to traditional unimodal methods, offering a new direction for FBP research. Furthermore, the model can be applied in various scenarios, such as: simulation postoperative assessment of personalized cosmetic surgery in the medical aesthetics; selection of optimal facial aesthetic enhancement solutions on social media; and recommendation of matching solutions in cosmetic recommendation.