Opt_att-GANC: Fusion of CT and PET images using an optimal attention-based generative adversarial network with classifier for lung cancer classification

Abstract

Lung cancer incidence and mortality continue to rise rapidly around the world. According to the American Cancer Society, the five-year survivability for individuals in the metastasis phases is significantly lower, highlighting the importance of early lung cancer diagnosis for effective therapy and improved quality of life. To achieve this, it is crucial to combine PET’s sensitivity for recognizing abnormal regions with CT’s anatomical localization for evaluating PET-CT images in computer-assisted detection implementations. Current PET-CT image evaluation methods either run each modality independently or aggregate the data from both, but they often overlook the fact that different visual features encode different types of data from different modalities. For instance, high atypical PET uptake within the lungs is more crucial for identifying tumors compared to physical PET uptake in the heart. To address the challenges of fine-grained issues during feature extraction and fusion, we propose an interpretable deep learning-based solution for lung cancer diagnosis using CT and PET images. This involves building an Optimal Adversarial Network for merging and an Optimal Attention-based Generative Adversarial Network with Classifier (Opt_att-GANC) to augment the classification of the existence and nonexistence of lung cancer based on extracted features. The performance of the Opt_att-GANC is compared with existing methodologies like global-feature encoding U-Net (GEU-Net), 3D Dense-Net, and 3D Convolutional Neural Network Technique (3D-CNN). Results show that the proposed Opt_att-GANC achieves an F1-score of 67.08%, 93.74% accuracy, 92% precision, 92.1% recall, and 93.74% recall. The prospective study aims to enhance the precision degree with reduced duration by incorporating an ensemble neural network paradigm for feature extraction.