Evaluation of an artificial intelligence generated Brock score to determine malignancy risk of screen-detected pulmonary nodules

Abstract

Background

Estimation of malignancy risk is a crucial step in the evaluation of indeterminate pulmonary nodules (IPN) on computed tomography (CT) scans. Artificial intelligence (AI) tools can improve the risk prediction of IPN. The purpose of this study was to compare the performance of an AI generated Brock score with standard Brock score to estimate malignancy risk.

Methods

The AI tool qCT (Qure.ai Technologies, Mumbai, India) was tested on a case-control series of patients with screen-detected nodules from the National Lung Cancer Screening Trial (NLST). Standard Brock score and AI Brock score were calculated for each nodule using clinical data from the NLST dataset and nodule characteristics derived from the dataset and AI interpretation of CT images, respectively. Performance of each model was assessed using the area under the receiver operating curve (AUC). The sensitivity and specificity of each model was calculated at fixed thresholds of ≥ 2, 5, and 10% cancer risk.

Results

A total of 478 patients (239 cases, 239 controls) were included. The AUC of the AI Brock score and standard Brock score were 0.72 (95% CI 0.67–0.76) and 0.76 (95% CI 0.72–0.81), respectively (p = 0.053). At a 5% threshold, the sensitivity and specificity were 69% and 65% for AI and 57% and 82% for the standard.

Conclusion

The accuracy of the AI tool was lower than the standard Brock score, though not reaching statistical significance. Across all risk thresholds, the AI tool had higher sensitivity, but lower specificity than the standard Brock score, suggesting use might complement clinician assessments of nodule risk.