The Role of Claude 3.5 Sonet and ChatGPT-4 in Posterior Cervical Fusion Patient Guidance.

Abstract

Background: This study evaluates the role of ChatGPT-4 and Claude 3.5 Sonet in postoperative management for patients undergoing posterior cervical fusion. It focuses on their ability to provide accurate, clear, and relevant responses to patient concerns, highlighting their potential as supplementary tools in surgical aftercare.

Methods: Ten common postoperative questions were selected and posed to ChatGPT-4 and Claude 3.5 Sonet. Ten independent neurosurgeons evaluated responses using a structured framework that assessed accuracy, response time, clarity, and relevance. A 5-point Likert scale also measured satisfaction, quality, performance, and importance. Advanced statistical analyses were used to compare the two AI platforms, including sensitivity, specificity, p-values, confidence intervals, and Cohen's d.

Results: Claude 3.5 Sonet outperformed ChatGPT-4 across all metrics, particularly in accuracy (96.5% vs. 80.6%), response time (92.9% vs. 76.4%), clarity (94.6% vs. 75.4%), and relevance (95.5% vs. 74.0%). Likert scale evaluations showed significant differences (p < 0.001) in satisfaction, quality, and performance, with Claude achieving higher ratings. Statistical analyses confirmed large effect sizes, high inter-rater reliability (kappa: 0.85-0.92 for Claude), and narrower confidence intervals, reinforcing Claude's consistency and superior performance.

Conclusion: Claude 3.5 Sonet demonstrated exceptional capability in addressing postoperative concerns for posterior cervical fusion patients, surpassing ChatGPT-4 in accuracy, clarity, and practical relevance. These findings underscore its potential as a reliable AI tool for enhancing patient care and satisfaction in surgical aftercare.