Using natural language processing to identify emergency department patients with incidental lung nodules requiring follow-up.

Abstract

Objectives: For emergency department (ED) patients, lung cancer may be detected early through incidental lung nodules (ILNs) discovered on chest CTs. However, there are significant errors in the communication and follow-up of incidental findings on ED imaging, particularly due to unstructured radiology reports. Natural language processing (NLP) can aid in identifying ILNs requiring follow-up, potentially reducing errors from missed follow-up. We sought to develop an open-access, three-step NLP pipeline specifically for this purpose.

Methods: This retrospective used a cohort of 26,545 chest CTs performed in three EDs from 2014 to 2021. Randomly selected chest CT reports were annotated by MD raters using Prodigy software to develop a stepwise NLP "pipeline" that first excluded prior or known malignancy, determined the presence of a lung nodule, and then categorized any recommended follow-up. NLP was developed using a RoBERTa large language model on the SpaCy platform and deployed as open-access software using Docker. After NLP development it was applied to 1000 CT reports that were manually reviewed to determine accuracy using accepted NLP metrics of precision (positive predictive value), recall (sensitivity), and F1 score (which balances precision and recall).

Results: Precision, recall, and F1 score were 0.85, 0.71, and 0.77, respectively, for malignancy; 0.87, 0.83, and 0.85 for nodule; and 0.82, 0.90, and 0.85 for follow-up. Overall accuracy for follow-up in the absence of malignancy with a nodule present was 93.3%. The overall recommended follow-up rate was 12.4%, with 10.1% of patients having evidence of known or prior malignancy.

Conclusions: We developed an accurate, open-access pipeline to identify ILNs with recommended follow-up on ED chest CTs. While the prevalence of recommended follow-up is lower than some prior studies, it more accurately reflects the prevalence of truly incidental findings without prior or known malignancy. Incorporating this tool could reduce errors by improving the identification, communication, and tracking of ILNs.