Iteratively refined ChatGPT outperforms clinical mentors in generating high-quality interprofessional education clinical scenarios: a comparative study.

Background: Interprofessional education (IPE) is essential for promoting teamwork among healthcare professionals. However, its implementation is often hindered by the limited availability of interprofessional faculty and scheduling challenges in creating high-quality IPE scenarios. While AI tools like ChatGPT are increasingly being explored for this purpose, they have yet to demonstrate the ability to generate high-quality IPE scenarios, which remains a significant challenge. This study examines the effectiveness of GPT-4o, an advanced version of ChatGPT enhanced by novel methodologies, in overcoming these obstacles.

Methods: This comparative study assessed clinical scenarios generated by GPT-4o using two strategies-standard prompt (a single-step scenario generation without iterative feedback) and iterative refinement (a multi-step, feedback-driven process)-against those crafted by clinical mentors. The iterative refinement method, inspired by actual clinical scenario development, employs a cyclical process of evaluation and refinement, closely mimicking discussions among professionals. Scenarios were evaluated for time efficiency and quality using the Interprofessional Quality Score (IQS), defined as the mean score assigned by multidisciplinary evaluators across five interprofessional criteria: clinical authenticity, team collaboration, educational alignment, appropriate challenge, and student engagement.

Results: Scenarios developed using the iterative refinement strategy were completed significantly faster than those by clinical mentors and achieved higher or equivalent IQS. Notably, these scenarios matched or exceeded the quality of those created by humans, particularly in areas such as appropriate challenge and student engagement. Conversely, scenarios generated via the standard prompt method exhibited lower accuracy and various other deficiencies. Blinded attribution assessments by students further demonstrated that scenarios developed through iterative refinement were often indistinguishable from those created by human mentors.

Conclusions: Employing GPT-4o with iterative refinement and role-playing strategies produces clinical scenarios that, in some areas, exceed those developed by clinical mentors. This approach reduces the need for extensive faculty involvement, highlighting AI's potential to closely align with established educational frameworks and substantially enhance IPE, particularly in resource-constrained settings.