A Retrospective External Validation of the Cleveland Clinic Malignancy Probability Prediction Model for Indeterminate Pulmonary Nodules

Abstract

Objective

To perform a retrospective, multicenter, external validation of the Cleveland Clinic malignancy probability prediction model for incidental pulmonary nodules.

Patients and Methods

From July 1, 2022, to May 31, 2023, we identified 296 patients who underwent tissue acquisition at Mayo Clinic (MC) (n=198) and Loyola University Medical Center (n=98) with histopathology indicating malignant (n=195) or benign (n=101). Data was collected at initial radiographic identification (point 1) and at the time of intervention (point 2). Point 3 represented the most recent data. The areas under the receiver operating characteristics were calculated for each model per time point. Calibration was evaluated by comparing the predicted and observed rates of malignancy.

Results

The areas under the receiver operating characteristics at time points 1, 2, and 3 for the MC model were 0.67 (95% CI, 0.61-0.74), 0.67 (95% CI, 0.58-0.77), and 0.70 (95% CI, 0.63-0.76), respectively. The Cleveland Clinic model (CCM) was 0.68 (95% CI, 0.61-0.74), 0.75 (95% CI, 0.65-0.84), and 0.72 (95% CI, 0.66-0.78), respectively. The mean ± SD estimated probability for malignant pulmonary nodules (PNs) at time points 1, 2, and 3 for the CCM was 64.2±25.9, 65.8±24.0, and 64.7±24.4, which resembled the overall proportion of malignant PNs (66%). The mean estimated probability of malignancy for the MC model at each time point was 38.3±27.4, 36.2±24.4, and 42.1±27.3, substantially lower than the observed proportion of malignancies.

Conclusion

The CCM found discrimination similar to its internal validation and good calibration. The CCM can be used to augment clinical and shared decision-making when evaluating high-risk PNs.