SCLResNet and DSAF: A self-supervised contrastive learning and deep self-attention fusion-based multimodal network for predicting central lymph node metastasis in papillary thyroid carcinoma

Accurate prediction of central lymph node metastasis (CLNM) in papillary thyroid carcinoma (PTC) is crucial to avoid unnecessary invasive procedures, yet existing models often fall short. We constructed the SCLResNet101 model based on a contrastive learning framework to extract network features of tumor ultrasound (US). SeResnet101 was used to extract network features of peri-vascular adipose tissue (PVAT) from the computed tomography (CT) of C6 (the arterial and venous layers beneath the thyroid). Univariate and multivariate analyses were performed using binary logistic regression to select clinical features. Finally, we constructed a Deep Self-Attention Fusion (DSAF) network to integrate features from these three modalities for CLNM prediction. Univariate and multivariate analyses revealed that Gender, Age, Size of US, and Extrathyroidal Extension (ETE) were independent risk factors for CLNM. In the internal test cohort (I-T), the area under the curve (AUC) of model was 0.863 (95 % CI: 0.779–0.932). In the external test cohort (E-T), the AUC was 0.839 (95 % CI: 0.755–0.905). Compared to all radiologists, the model significantly reduced both false-positive and false-negative rates in both the I-T and E-T. This study incorporates PVAT, which significantly enhances the performance of the multimodal deep learning model and may assist surgeons in making more informed and precise surgical decisions in the treatment of PTC.