Enhancing GP consultation skills training: educational evaluation of a conversational AI innovation for simulated consultation assessment preparation.

Abstract

Background: Simulated Consultation Assessment (SCA) preparation is critical for GP trainees, with the traditional approach relying on trained medical actors. However, limited actor availability, scheduling constraints and high costs often restrict practice opportunities. This evaluation aimed to assess a conversational artificial intelligent (AI) system as an educational tool for GP consultation skills training, examining its acceptability, perceived educational impact and implementation considerations for training.

Methods: We conducted an educational innovation evaluation with GP trainees and educators (n = 22) following implementation of a conversational AI simulation system for SCA preparation. The evaluation employed a mixed-methods design incorporating validated educational assessment measures across four domains: clinical authenticity, educational utility, user experience and technical performance. Participants' perspectives were captured through structured questionnaires and open-ended feedback. Implementation considerations were assessed through cost-comparison analysis with traditional training methods.

Results: This educational innovation demonstrated strong acceptability among users, with particularly positive evaluations for clinical content authenticity (median 4.5, IQR 4-5) and educational value (median 4.5, IQR 4-5). Qualitative feedback revealed that the innovation successfully addressed key training needs, particularly around accessibility and practice frequency. More experienced educators (11+ years) rated the innovation significantly higher for educational utility (p < 0.05), suggesting potential for curriculum integration. The implementation analysis revealed resource advantages, with 24-84% cost reductions compared to traditional methods.

Conclusion: This conversational AI innovation offers GP educators a practical, cost-effective solution to training challenges, potentially addressing the modality mismatch between face-to-face training and video-based examinations. Implementation should focus on integration as a complementary curriculum tool, with ongoing evaluation of educational outcomes.