TonguExpert: A Deep Learning-Based Algorithm Platform for Fine-Grained Extraction and Classification of Tongue Phenotypes.

Abstract

Tongue analysis holds promise for disease detection and health monitoring, especially in traditional Chinese medicine. However, its subjectivity hinders clinical applications. Deep learning offers a path for automated tongue diagnosis, yet existing methods struggle to capture subtle details, and the lack of large datasets hampers the development of robust and generalizable models. To address these challenges, we introduce TonguExpert (https://www.biosino.org/TonguExpert), a free platform for archiving, analyzing, and extracting phenotypes from tongue images. Our deep learning framework integrates cutting-edge techniques for tongue segmentation and phenotype extraction. TonguExpert analyzes a massive dataset of 5992 tongue images from a Chinese population and extracts 773 phenotypes including five predicted labels and their probabilities, 355 global features (entire tongue, tongue body, and tongue coating) and 408 local features (fissures and tooth marks) in a unified process. Besides, 580 additional features for five tongue subregions are also available for future study. Notably, TonguExpert outperforms manual classification methods, achieving high accuracy (ROC-AUC 0.89-0.99 for color, 0.97 for fissures, 0.88 for tooth marks). Additionally, the model generalizes well to predict new phenotypes (e.g., greasy coating) using external datasets. This allows the model to learn from a broader spectrum of data, potentially improving its overall performance. We also release the largest publicly available dataset of tongue images and phenotypes, which is invaluable for advancing automated analysis and clinical applications of tongue diagnosis. In summary, this research advances automated tongue diagnosis, paving the way for wider clinical adoption and potentially expanding the applications in the future.

Supplementary information: The online version contains supplementary material available at 10.1007/s43657-024-00210-9.