Performance of ChatGPT-4o in thoracic trauma: A comparative evaluation with guidelines.

Abstract

Background: This study aims to evaluate the performance of ChatGPT-4o in thoracic trauma management by comparing its responses to established clinical guidelines.

Methods: Five major thoracic surgery guidelines were reviewed, including the Advanced Trauma Life Support (ATLS) Guidelines 2018, Eastern Association for the Surgery of Trauma (EAST) Guidelines 2020, Evaluation and management of traumatic pneumothorax: A Western Trauma Association critical decisions algorithm 2022, European Trauma Course (ETC) Guidelines 2016, and the National Institute for Health and Care Excellence (NICE) Guidelines for Trauma 2020. Fifty open-ended questions were developed based on these guidelines and submitted to ChatGPT-4o. Five thoracic surgery specialists evaluated the artificial intelligence (AI) responses using a 5-point Likert scale.

Results: ChatGPT-4o achieved an average score of 4.76+-0.57 on the 50-question evaluation. ChatGPT-4o excelled in questions derived from well-defined guidelines, demonstrating its ability to synthesize and apply guideline-based medical knowledge. Its performance aligns with previous studies in urological trauma and emergency medicine, which reported similar reliability. However, its reliance on pre-existing data limits its effectiveness in addressing highly nuanced or novel clinical scenarios. These findings underscore its potential as a complementary tool in guideline-driven medical contexts while emphasizing the need for clinical oversight in complex cases.

Conclusion: ChatGPT-4o performed strongly in thoracic trauma management questions, demonstrating minimal errors and high reliability. These results suggest it could serve as a valuable support tool for clinical decision-making, particularly in scenarios guided by established protocols. Further exploration into broader medical domains is warranted.