Not from scratch: Explainable deep transfer learning fine-tunning with domain adaptation enables trustworthy COVID-19 prediction

Abstract

Medical image analysis can help diagnose Coronavirus Disease 2019 (COVID-19) early and save patient lives before the disease worsens. However, there are various limitations to manual inspection of these medical images, such as dependence on physician experience and subjectivity of assessment. To enable fast and precise disease diagnosis, we propose XDTLMI-Net, a framework using four CNNs (GoogLeNet, ResNet18, ResNet50, ResNet101) skilled in image data processing. This framework uses existing medical domain knowledge to guide transfer learning for COVID-19 Computed tomography (CT) scan images and Chest X-rays (CXR) images. XDTLMI-Net performed three tasks of medical image classification of COVID-19 on three public datasets: COVID-19 CT, SARS-COV-2 CT and COVID-19 CXR. It achieved an average classification accuracy of 0.9897, 0.9752 and 0.9397, and an average classification F1-score of 0.9 guide transfer learning with 898, 0.9741 and 0.9394, respectively. Moreover, we employed the Shaply Additive exPlanations and Gradient-weighted Class Activation Mapping to interpret the COVID-19 predictions and help understand the predictive models’ decision-making process. Generally, a general end-to-end framework called XDTLMI-Net based on CNN and transfer learning was developed, which works on small datasets of medical images, and does not require any segmentation or image preprocessing procedures. Moreover, XDTLMI-Net outperformed on three datasets in fine-tuning course and gave reasonable importance to each input COVID-19 image, showing its potential for application in different clinical scenarios.