Development and Multi-center validation of a machine learning Model for advanced colorectal neoplasms screening

Abstract

Background

In colorectal cancer (CRC) screening programs, accurately identifying individuals at high risk for advanced colorectal neoplasia (ACN) is essential as they require further colonoscopy, early intervention, and monitoring follow-up. This study aimed to develop a machine learning (ML)-based risk prediction model, serving as an effective tool for the early identification of high-risk individuals for ACN.

Methods

This study analyzed data from the Tianjin CRC screening program. The dataset from 2012 to 2022 was divided into a training set and 11 validation sets across 12 medical centers. The 2023 data was used as an independent temporal external validation set. First, the least absolute shrinkage and selection operator and logistic regression (LR) analysis were used to select significant features. Next, six ML models were constructed to predict ACN and validate its predictive capability on the validation sets. Among the classifiers, the best-performing model, Tianjin ML (TML), was selected, and its performance was compared with the Asia Pacific Colorectal Screening (APCS) and LR. Moreover, we developed a stacked ensemble learning model to improve the prediction performance for ACN. Finally, we conducted an interpretability analysis using SHapley Additive exPlanations (SHAP) values and deployed a web application tool based on the Streamlit framework.

Findings

Among the trained models, the TML achieved the best performance, with an area under the curve (AUC) of 0.690, a sensitivity of 0.649, a specificity of 0.626, an F1 score of 0.320, and an accuracy of 0.629. Furthermore, the TML performed well in 11 validation sets and the independent temporal external validation set. A predictive probability threshold of 0.140 was identified for stratifying individuals into low- and high-risk groups. The TML exhibited superior performance compared to APCS and LR. The stacked ensemble learning model, S-TML, further improved the AUC to 0.709. Additionally, SHAP analysis identified age, gender, and fecal immunochemical tests as the top three predictive factors for ACN.

Interpretation

The TML outperformed traditional models, including APCS and LR, in predicting ACN and could serve as a screening decision support tool to identify high–risk individuals for ACN.