Improving Colorectal Cancer Screening and Risk Assessment through Predictive Modeling on Medical Images and Records.

Colonoscopy screening effectively identifies and removes polyps before they progress to colorectal cancer (CRC), but current follow-up guidelines rely primarily on histopathological features, overlooking other important CRC risk factors. Variability in polyp characterization among pathologists also hinders consistent surveillance decisions. Advances in digital pathology and deep learning enable the integration of pathology slides and medical records for more accurate progression risk prediction. Using data from the New Hampshire Colonoscopy Registry, including longitudinal follow-up, a transformer-based model for histopathology image analysis was adapted to predict 5-year progression risk. Multi-modal fusion strategies were further explored to combine clinical records with deep learning-derived image features. Training the model to predict intermediate clinical variables improved 5-year progression risk prediction (AUC = 0.630) compared to direct prediction (AUC = 0.615, p = 0.013). Integrating WSI-based model predictions with non-imaging features further improved performance (AUC = 0.672), significantly outperforming the non-imaging-only approach (AUC = 0.666, p = 0.002). These results highlight the value of integrating diverse data modalities with computational methods to enhance progression risk stratification.