Multi-DeepNet: A Novel Weakly-Supervised Multi-Task and Multi-View-Oriented Convolution Neural Network for COVID-19 Diagnosis from CT Images

Abstract

Currently, manual analysis performed by professional radiologists is required for COVID-19 diagnosis given the patient's chest Computed Tomography (CT) images, but this process is inefficient and costly. Deep learning methods can provide computer vision-based solutions to help guide radiologists perform faster and more accurate diagnosis. However, current well performed methods require training on large and balanced datasets with pixel level lung lesion annotations, both of which are not easily accessible. Moreover, visual similarities between COVID-19 and other pneumonia in CT scans make it difficult to learn their distinguishing features. To address these issues, we propose a novel weakly-supervised deep learning model, named Multi-DeepNet, that can be well trained to perform fine-grained classification on small and imbalanced datasets. Specifically, a multi-task pre-training module is introduced to better extract distinguishing features between COVID-19 and other similar pneumonia. Furthermore, a multi-view-oriented classifier is proposed to extract complimentary information from the axial, coronal and sagittal planes. Experimental results demonstrate that our Multi-DeepNet achieves superior sensitivities, specificity, and accuracies compared to state-of-the-art methods.