Computer-assisted diagnosis for axillary lymph node metastasis of early breast cancer based on transformer with dual-modal adaptive mid-term fusion using ultrasound elastography
Chihao Gong , Yinglan Wu , Guangyuan Zhang , Xuan Liu , Xiaoyao Zhu , Nian Cai , Jian Li
{"title":"Computer-assisted diagnosis for axillary lymph node metastasis of early breast cancer based on transformer with dual-modal adaptive mid-term fusion using ultrasound elastography","authors":"Chihao Gong , Yinglan Wu , Guangyuan Zhang , Xuan Liu , Xiaoyao Zhu , Nian Cai , Jian Li","doi":"10.1016/j.compmedimag.2024.102472","DOIUrl":null,"url":null,"abstract":"<div><div>Accurate preoperative qualitative assessment of axillary lymph node metastasis (ALNM) in early breast cancer patients is crucial for precise clinical staging and selection of axillary treatment strategies. Although previous studies have introduced artificial intelligence (AI) to enhance the assessment performance of ALNM, they all focus on the prediction performances of their AI models and neglect the clinical assistance to the radiologists, which brings some issues to the clinical practice. To this end, we propose a human–AI collaboration strategy for ALNM diagnosis of early breast cancer, in which a novel deep learning framework, termed DAMF-former, is designed to assist radiologists in evaluating ALNM. Specifically, the DAMF-former focuses on the axillary region rather than the primary tumor area in previous studies. To mimic the radiologists’ alternative integration of the UE images of the target axillary lymph nodes for comprehensive analysis, adaptive mid-term fusion is proposed to alternatively extract and adaptively fuse the high-level features from the dual-modal UE images (i.e., B-mode ultrasound and Shear Wave Elastography). To further improve the diagnostic outcome of the DAMF-former, an adaptive Youden index scheme is proposed to deal with the fully fused dual-modal UE image features at the end of the framework, which can balance the diagnostic performance in terms of sensitivity and specificity. The clinical experiment indicates that the designed DAMF-former can assist and improve the diagnostic abilities of less-experienced radiologists for ALNM. Especially, the junior radiologists can significantly improve the diagnostic outcome from 0.807 AUC [95% CI: 0.781, 0.830] to 0.883 AUC [95% CI: 0.861, 0.902] (<span><math><mi>P</mi></math></span>-value <span><math><mo><</mo></math></span>0.0001). Moreover, there are great agreements among radiologists of different levels when assisted by the DAMF-former (Kappa value ranging from 0.805 to 0.895; <span><math><mi>P</mi></math></span>-value <span><math><mo><</mo></math></span>0.0001), suggesting that less-experienced radiologists can potentially achieve a diagnostic level similar to that of experienced radiologists through human–AI collaboration. This study explores a potential solution to human–AI collaboration for ALNM diagnosis based on UE images.</div></div>","PeriodicalId":50631,"journal":{"name":"Computerized Medical Imaging and Graphics","volume":"119 ","pages":"Article 102472"},"PeriodicalIF":5.4000,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computerized Medical Imaging and Graphics","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0895611124001496","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Accurate preoperative qualitative assessment of axillary lymph node metastasis (ALNM) in early breast cancer patients is crucial for precise clinical staging and selection of axillary treatment strategies. Although previous studies have introduced artificial intelligence (AI) to enhance the assessment performance of ALNM, they all focus on the prediction performances of their AI models and neglect the clinical assistance to the radiologists, which brings some issues to the clinical practice. To this end, we propose a human–AI collaboration strategy for ALNM diagnosis of early breast cancer, in which a novel deep learning framework, termed DAMF-former, is designed to assist radiologists in evaluating ALNM. Specifically, the DAMF-former focuses on the axillary region rather than the primary tumor area in previous studies. To mimic the radiologists’ alternative integration of the UE images of the target axillary lymph nodes for comprehensive analysis, adaptive mid-term fusion is proposed to alternatively extract and adaptively fuse the high-level features from the dual-modal UE images (i.e., B-mode ultrasound and Shear Wave Elastography). To further improve the diagnostic outcome of the DAMF-former, an adaptive Youden index scheme is proposed to deal with the fully fused dual-modal UE image features at the end of the framework, which can balance the diagnostic performance in terms of sensitivity and specificity. The clinical experiment indicates that the designed DAMF-former can assist and improve the diagnostic abilities of less-experienced radiologists for ALNM. Especially, the junior radiologists can significantly improve the diagnostic outcome from 0.807 AUC [95% CI: 0.781, 0.830] to 0.883 AUC [95% CI: 0.861, 0.902] (-value 0.0001). Moreover, there are great agreements among radiologists of different levels when assisted by the DAMF-former (Kappa value ranging from 0.805 to 0.895; -value 0.0001), suggesting that less-experienced radiologists can potentially achieve a diagnostic level similar to that of experienced radiologists through human–AI collaboration. This study explores a potential solution to human–AI collaboration for ALNM diagnosis based on UE images.
期刊介绍:
The purpose of the journal Computerized Medical Imaging and Graphics is to act as a source for the exchange of research results concerning algorithmic advances, development, and application of digital imaging in disease detection, diagnosis, intervention, prevention, precision medicine, and population health. Included in the journal will be articles on novel computerized imaging or visualization techniques, including artificial intelligence and machine learning, augmented reality for surgical planning and guidance, big biomedical data visualization, computer-aided diagnosis, computerized-robotic surgery, image-guided therapy, imaging scanning and reconstruction, mobile and tele-imaging, radiomics, and imaging integration and modeling with other information relevant to digital health. The types of biomedical imaging include: magnetic resonance, computed tomography, ultrasound, nuclear medicine, X-ray, microwave, optical and multi-photon microscopy, video and sensory imaging, and the convergence of biomedical images with other non-imaging datasets.