Can ChatGPT detect breast cancer on mammography?

Abstract

Some noteworthy studies have questioned the use of ChatGPT, a free artificial intelligence program that has become very popular and widespread in recent times, in different branches of medicine. In this study, the success of ChatGPT in detecting breast cancer on mammography (MMG) was evaluated. The pre-treatment mammographic images of patients with a histopathological diagnosis of invasive breast carcinoma and prominent mass formation on MMG were read separately into two ChatGPT subprograms: Radiologist Report Writer (P1) and XrayGPT (P2). The programs were asked to determine mammographic breast density, tumor size, side, and quadrant, the presence of microcalcification, distortion, skin or nipple changes, and axillary lymphadenopathy (LAP), and BI-RADS score. The responses were evaluated in consensus by two experienced radiologists. Although the mass detection rate of both programs was over 60%, the success in determining breast density, tumor size and localization, microcalcification, distortion, skin or nipple changes, and axillary LAP was low. BI-RADS category agreement with readers was fair for P1 (κ:28%, 0.20< κ ≤ 0.40) and moderate for P2 (κ:58%, 0.40< κ ≤ 0.60). In conclusion, while the XrayGPT application can detect breast cancer with a mass appearance on MMG images better than the Radiologist Report Writer application, the success of both is low in detecting all other related features. This casts doubt over the suitability of current large language models for image analysis in breast screening.