Assessment of Appearance-related Questions About Breast Reconstruction Generated by Chat Generative Pre-trained Transformer.

Abstract

Background: Previous studies have explored the ability of artificial intelligence (AI) tools based on large language models, such as Chat Generative Pre-Trained Transformer (ChatGPT), to answer patient questions about breast reconstruction. In this study, we assessed the quality of questions generated by ChatGPT for breast reconstruction patients to ask their providers.

Methods: ChatGPT was prompted to generate appearance-related questions representative of what patients might ask during breast reconstruction consultations. As a benchmark, a comparison group of questions from credible online sources was compiled. Blinded to the source, surgeons assessed the quality of questions in terms of their acceptability, contribution to the informed consent process, and contribution to the shared decision-making process. Surgeons were also asked to report their agreement on whether questions were generated by an AI tool.

Results: Experienced reconstructive surgeons rated ChatGPT-generated questions about appearance-related outcomes of breast reconstruction as acceptable (15 of 16), likely to positively contribute to the informed consent process (15 of 16), and likely to positively contribute to the shared decision-making process (16 of 16). These ratings were comparable to those for benchmark questions. Surgeons did not readily recognize questions as being AI-generated. Differences in surgeon assessments were most pronounced regarding the perceived potential for the questions to contribute to the informed consent process.

Conclusions: The quality of ChatGPT-generated questions related to appearance concerns is comparable to that of questions sourced from reputable online websites. Patients may benefit from discussion with their providers about best practices for using AI tools in preparation for consultations.