Rectal-RadioSAM: Large model-assisted multi-parametric magnetic resonance imaging pipeline for predicting response to neoadjuvant chemoradiotherapy in rectal cancer without human intervention

Background and purpose

Accurate evaluation of response to neoadjuvant chemoradiotherapy (nCRT) in rectal cancer is critical for guiding clinical decision-making. This study developed and validated a large model-assisted automated prediction tool to assess response to nCRT in locally advanced rectal cancer (LARC), focusing on segmentation and radiomic feature extraction.

Material and methods

A retrospective analysis included 378 LARC patients (756 cases: baseline and post-nCRT MRI). MRI protocols comprised T2-weighted imaging (T2WI) and logarithmic diffusion-weighted imaging (DWI, b = 1000 s/mm²). A two-stage hybrid model combined fine-tuned four-channel MedSAM networks for lesion segmentation and a coupled XGBoost model for pathologic complete response (pCR) prediction. Resilience of radiomic features was assessed by comparing automated and manual segmentations.

Results

In the independent testing set comprising 112 LARC patients, the large segmentation models achieved mean (± std) Dice coefficients of 0.74 (± 0.09), 0.66 (± 0.13), 0.67 (± 0.15), and 0.46 (± 0.15) for pre-nCRT T2WI, post-nCRT T2WI, pre-nCRT DWI (log[S(1000)]), and post-nCRT DWI (log[S(1000)]) images, respectively. Meanwhile, First-Order and Shape radiomic features exhibited significant correlations between the large model-assisted segmentations and manual delineations (p < 0.01). In the prediction phase, the combined pipeline achieved a mean (± std) AUC value of 0.83 (± 0.04).

Conclusion

The large model-assisted multi-parametric MRI pipeline demonstrated robust performance in predicting pCR for rectal cancer, enabling fully automated radiological assessment without human intervention.