Classifying Tongue Images using Deep Transfer Learning

Traditional Chinese Medicine (TCM) believes that the tongue image is closely related to the health of the human organs and tongues’ visual features can provide valuable clues for disease diagnosis. Applying tongue image analysis technique for automatic disease diagnosis is an active research filed in the modernization of TCM. Although deep learning has advantages over traditional methods in automatic extraction of high-dimensional features, it needs large training samples, which limits its application in medical image analysis, especially in tongue image, because it is difficult to collect enough labeled images. In this paper, we make the first attempt to use deep transfer learning for tongue image analysis. First, we extract the tongue features through the pre-trained networks (ResNet and Inception_v3), and then rewrite the output layer of the original network with global average pooling and full-connected layer to output classification results. A dataset of 2245 tongue images we collected from specialized TCM medical institutions is used for classification performance evaluation. The experimental results demonstrate that the proposed method achieves the better classification accuracy than the existing deep learning methods which proves the effectiveness of the proposed deep transfer learning for tongue image classification.