Artificial Intelligence for early detection of lung cancer in General Practitioners' clinical notes.

Abstract

Background: The journey of more than 80% of patients diagnosed with lung cancer starts in general practice. About 75% of patients are diagnosed in an advanced stage (3 or 4), leading to more than 80% mortality within one year at present. The long-term data in general practitioners' records might contain hidden information that could be used for earlier case-finding of patients with cancer.

Aim: To develop new prediction tools that improve the risk assessment for cancer.

Design and setting:  Text analysis of electronic patient data using natural language processing and machine learning in general practice files of four networks in the Netherlands.

Method: Files of 525,526 patients were analysed, of whom 2386 were diagnosed with lung cancer. Diagnoses were validated in the Dutch Cancer registration, and structured and free text data were used to predict diagnosis of lung cancer five months before diagnosis (four months before referral).

Results: Our algorithm could facilitate earlier detection of lung cancer using routine general practice data. We established discrimination, calibration, sensitivity, and specificity under various cut-off points of the prediction five months before diagnosis. Internal validation demonstrated an area under the curve of 0.90 (CI 95%: 0.90-0.93), and 0.84 (CI: 0.83-0.85) during external validation. The desired sensitivity determines the number of patients to be referred to detect one patient with lung cancer.

Conclusion:  AI-based support enables earlier detection of lung cancer in general practice using readily available text in the patient files of general practitioners, but needs additional prospective clinical evaluation.