Pulmonary Nodule Classification with Multi-View Convolutional Vision Transformer

Abstract

Pulmonary nodule classification from computerized tomography(CT) Scans is a vital task for the early screening of Lung cancers. The algorithm is aiming at distinguishing malignant pulmonary nodules, benign nodules and the ones with their subtypes. In this paper, we defined a detailed pulmonary nodule classification task considering 5 semantic labels. We are facing with a series of non-trival problems dealing with such a task. First, the available medical image data for training is quite limited. We enlarged the training dataset by cropping out three-dimension(3D) volume of each pulmonary nodule and generating 15 planes with different orientations from these volumes. Secondly, the global modeling ability of the existing convolutional neural network(CNN) based architectures can not meet the need of medical image analysis well. To learn discriminative abstract information, we down-sample feature maps between successive stages and adopt the BotNet-50 backbone which is a combination of ResNet backbone and self-attention modules. Such an architecture can extract local and non-local information in low-level and high-level layers, respectively. Last but not the least, the data distribution of training data and testing data don't share similar distribution in real-world multi-center medical image classification scenes. We assigned the samples with modified wights while calculating the loss value for optimization. The proposed method can eliminate the spurious correlation between features and labels. Experiments demonstrate the effectiveness of each component.