A Pilot Study Assessing the Accuracy of AI ChatGPT Responses for AL Amyloidosis

Abstract

AL amyloidosis is a rare, complex and often challenging disease for both patients and healthcare providers. The availability of accurate medical information is crucial for effective diagnosis and management. In recent years, artificial intelligence (AI) has emerged as a potential tool for providing medical information. This study aims to assess the accuracy of AI ChatGPT responses for AL amyloidosis related common questions and compare them to expert opinions. A scoring system was developed to evaluate responses provided by five participating expert physicians. AI ChatGPT demonstrated an overall accuracy rate of 82% in answering AL amyloidosis-related questions. Responses on prognosis and patient support received the highest scores (100%), while questions related to treatment options showed lower accuracy (30%–60%). The results indicate that while the AI ChatGPT demonstrates overall accuracy, there are areas for improvement and potential discrepancies compared to expert opinions. These findings highlight the importance of ongoing refinement and validation of AI-powered medical tools and cannot yet replace the advice of experts in the disease.