A Study of Vision Transformer for Lung Diseases Classification

Transformer models have gained much success in natural language processing. In the computer vision field, transformer-based backbones recently compete with CNN-based backbones in many tasks. The success of transformer-based backbones relies on a pre-trained model that is trained on huge datasets. However, the requirement may not be satisfied in medical image applications. Compared to ImageNet 21K dataset, medical image datasets are very limited. Therefore, in this paper, we discover the performance of the Vision Transformer on medical image classification. The vision transformer will be fine-tuned on well-known medical datasets. Later, it will be fine-tuned again on the VinDr-CXR dataset. Comprehensive experiments show that the proposed method is slightly better than conventional convolution-based methods in terms of classification accuracy. However, in terms of model interpretability, ViT based models can handle the co-occurrence of multi-diseases in a medical image.