Facial Complexion Recognition of Traditional Chinese Medicine Based on Computer Vision

Abstract

This paper makes an attempt to develop an automated facial complexion recognition method for objective and quantitative facial diagnosis. In TCM diagnosis, some regions of the face like Ting, Jia and Mingtang, can provide the most valuable information, so we use deep learning technique to determine the 68 landmarks of face and use their location to segment the regions of interest (ROI). The statistical characteristics of color histograms in multiple color space and texture features, lip color features are then introduced to describe the facial complexion. Finally, several machine learning methods including KNN, SVM and BPNN are used for classification. To verify the validity of our method, we collected a dataset of 575 face images from professional TCM medical institutions. The results show that the process of ROIs’ segmentation can improve the accuracy efficiently, higher than unsegmented image. The proposed method by fusing all three features achieves an accuracy of 91.03% which is higher than the existing methods and proves the effectiveness of the proposed method for facial complexion recognition. We confirm that extracting the complexion features particularly from the regions of interest of the face image achieves higher classification accuracy than characterizing the overall complexion directly from the unsegmented images. We show that the facial color features provide the most important clues for complexion classification among all the used features, which is consistent with the TCM diagnosis. Finally, we prove that the facial texture feature and lip color feature can be used as complementary clues and fused with the facial color features for further improving the complexion classification accuracy.