External validation of the performance of commercially available deep-learning-based lung nodule detection on low-dose CT images for lung cancer screening in Japan.

Purpose: Artificial intelligence (AI) algorithms for lung nodule detection have been developed to assist radiologists. However, external validation of its performance on low-dose CT (LDCT) images is insufficient. We examined the performance of the commercially available deep-learning-based lung nodule detection (DL-LND) using LDCT images at Japanese lung cancer screening (LCS).

Materials and methods: Included were 43 patients with suspected lung cancer on LDCT images and pathologically confirmed lung cancer. The reference standard for nodules whose diameter exceeded 4 mm was set by a radiologist who referred to the reports of two other radiologists reading the LDCT images. After we applied commercially available DL-LND to the LDCT images, the radiologist reviewed all nodules detected by DL-LND. When he failed to identify an existing nodule, it was also included in the reference standard. To validate the performance of DL-LND, the sensitivity for lung nodules and lung cancer, the positive-predictive value (PPV) for lung nodules, and the mean number of false-positive (FP) nodules per CT scan were recorded.

Results: The radiologist detected 97 nodules including 43 lung cancers and missed 3 solid nodules detected by DL-LND. A total of 100 nodules was included in the reference standard. DL-LND detected 396 nodules including 40 lung cancers. The sensitivity for the 100 nodules was 96.0%; the PPV was 24.2% (96/396). The mean number of FP nodules per CT scan was 7.0; sensitivity for lung cancer was 93.0% (40/43). DL-LND missed three lung cancers; 2 of these were atypical pulmonary cysts.

Conclusion: We externally verified that the sensitivity for lung nodules and lung cancer by DL-LND was very high. However, its low PPV and the increased FP nodules remains a serious drawback of DL-LND.