Clinical transportability, calibration drift, and local updating of a B-mode ultrasound radiomics-clinical prediction model for occult high-volume central lymph node metastasis in cN0 papillary thyroid carcinoma: development, temporal validation, and external validation.

Abstract

Background: Occult high-volume central lymph node metastasis (CLNM) can remain undetected in clinically node-negative papillary thyroid carcinoma (cN0 PTC), yet nodal burden influences preoperative counseling and recurrence-oriented risk stratification. For clinical implementation, a preoperative radiomics-based risk estimate must remain transportable, locally calibrated, and interpretable after transfer to another center. This study evaluated the clinical transportability, calibration drift, threshold consequences, and local updating of a B-mode ultrasound radiomics-clinical prediction model for occult high-volume CLNM.

Methods: This three-center retrospective model-development, validation, and updating study used the source workbook containing 957 cN0 PTC patients. The endpoint was occult high-volume CLNM, defined as more than five metastatic central lymph nodes on postoperative pathology. Candidate predictors were restricted to preoperative clinical-ultrasound variables and a prespecified B-mode radiomics score. Clinical-only, B-mode score-only, and clinical plus B-mode score models were compared in the development cohort; the clinical plus B-mode score model was treated as the locked primary implementation model because it preserved routine clinical interpretability while incorporating the quantitative image signal. The locked model was evaluated in a temporal Center 1 cohort (n=210), Center 2 (n=139), and Center 3 (n=119). Discrimination, Brier score, calibration, fixed-threshold classification, decision curve analysis, threshold consequences, and post hoc intercept-only recalibration were assessed.

Results: High-volume CLNM was present in 399 of 957 patients (41.7%). Event rates were 44.8%, 45.2%, 41.0%, and 23.5% in the development, temporal validation, Center 2, and Center 3 cohorts. Development-cohort comparisons showed larger tumors, higher B-mode radiomics scores, higher Chinese Thyroid Imaging Reporting and Data System (C-TIRADS) subclass, and richer vascularity among endpoint-positive patients. The B-mode score accounted for most discrimination. Compared with the B-mode score-only model, adding clinical-ultrasound variables changed the area under the receiver operating characteristic curve (AUC) by +0.015 in development, +0.008 in temporal validation, -0.009 in Center 2, and -0.029 in Center 3. The locked clinical plus B-mode score model had AUC values of 0.823, 0.874, 0.866, and 0.870, respectively. Mean predicted risk aligned with observed risk in the development, temporal validation, and Center 2 cohorts but overestimated risk in Center 3 (36.4% predicted vs. 23.5% observed; calibration intercept, -0.902). At the locked threshold of 0.565, Center 3 retained specificity of 0.857 and negative predictive value (NPV) of 0.897, while intercept-only updating reduced the Center 3 Brier score from 0.138 to 0.121.

Conclusions: The locked radiomics-clinical model provided an interpretable preoperative risk-stratification framework whose deployment value depends on local calibration and threshold consequences. External deployment should evaluate spectrum shift, positive predictive value (PPV) and NPV, calibration, decision curves, and local updating rather than relying on AUC alone.