Matthew E Kelleher, C Y Zhou, Seth Overla, Andrew Zahn, Brandon Zaffuto, Sally A Santen, Benjamin Kinnear, Laurah Turner
{"title":"Building CAR-E: A Novel Artificial Intelligence Agent for Coaching Conversations.","authors":"Matthew E Kelleher, C Y Zhou, Seth Overla, Andrew Zahn, Brandon Zaffuto, Sally A Santen, Benjamin Kinnear, Laurah Turner","doi":"10.5334/pme.2563","DOIUrl":null,"url":null,"abstract":"<p><strong>Background and need for innovation: </strong>Coaching has many benefits in medical education, but large-scale implementation is constrained by faculty bandwidth, scheduling, cost, and power differentials. Artificial intelligence (AI), specifically large language models (LLMs), offers potential solutions to augment coaching, but limitations remain and refining performance is necessary to create valuable coaching conversations.</p><p><strong>Steps taken for development and implementation of innovation: </strong>The authors created Coaching with AI-Reinforced Education (CAR-E), an AI coaching agent built using an LLM. Using layered architecture, CAR-E couples real-time speech input and output, retrieval-augmented generation (RAG) for evidence-based coaching, and dual memory system (short-term context and long-term history) to sustain longitudinal dialogue. An iterative design refined CAR-E with users who voluntarily engaged in coaching conversations and provided feedback. Transcripts and user feedback were analyzed to improve CAR-E.</p><p><strong>Evaluation of innovation: </strong>During the pilot, 37 medical trainees and faculty engaged in coaching conversations. Transcripts showed diverse topics discussed and many strengths and opportunities for improvement. Users reported that CAR-E facilitated self-reflection, clarified goals, and broke complex problems into actionable steps. They also noted formulaic questioning that felt repetitive, superficial attempts to display empathy and difficulty moving conversations forward. Many users were frustrated by CAR-E's strict adherence to coaching competencies, which prevented it from giving advice or suggestions.</p><p><strong>Critical reflection on your process: </strong>Unlike generic LLMs that default to broad, solution-oriented dialogue, CAR-E was designed for coaching in medical education. Its design incorporates evidence-based knowledge retrieval with curated coaching resources, structured memory to support longitudinal conversation, institutional oversight, and speech integration. These features exemplify responsible AI use in medical education. Despite this architecture, many improvements are planned to create reflective, growth-oriented conversations that augment coaching in medical education.</p>","PeriodicalId":48532,"journal":{"name":"Perspectives on Medical Education","volume":"15 1","pages":"546-555"},"PeriodicalIF":3.5000,"publicationDate":"2026-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13308527/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Perspectives on Medical Education","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.5334/pme.2563","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/1/1 0:00:00","PubModel":"eCollection","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
引用次数: 0
Abstract
Background and need for innovation: Coaching has many benefits in medical education, but large-scale implementation is constrained by faculty bandwidth, scheduling, cost, and power differentials. Artificial intelligence (AI), specifically large language models (LLMs), offers potential solutions to augment coaching, but limitations remain and refining performance is necessary to create valuable coaching conversations.
Steps taken for development and implementation of innovation: The authors created Coaching with AI-Reinforced Education (CAR-E), an AI coaching agent built using an LLM. Using layered architecture, CAR-E couples real-time speech input and output, retrieval-augmented generation (RAG) for evidence-based coaching, and dual memory system (short-term context and long-term history) to sustain longitudinal dialogue. An iterative design refined CAR-E with users who voluntarily engaged in coaching conversations and provided feedback. Transcripts and user feedback were analyzed to improve CAR-E.
Evaluation of innovation: During the pilot, 37 medical trainees and faculty engaged in coaching conversations. Transcripts showed diverse topics discussed and many strengths and opportunities for improvement. Users reported that CAR-E facilitated self-reflection, clarified goals, and broke complex problems into actionable steps. They also noted formulaic questioning that felt repetitive, superficial attempts to display empathy and difficulty moving conversations forward. Many users were frustrated by CAR-E's strict adherence to coaching competencies, which prevented it from giving advice or suggestions.
Critical reflection on your process: Unlike generic LLMs that default to broad, solution-oriented dialogue, CAR-E was designed for coaching in medical education. Its design incorporates evidence-based knowledge retrieval with curated coaching resources, structured memory to support longitudinal conversation, institutional oversight, and speech integration. These features exemplify responsible AI use in medical education. Despite this architecture, many improvements are planned to create reflective, growth-oriented conversations that augment coaching in medical education.
期刊介绍：
Perspectives on Medical Education mission is support and enrich collaborative scholarship between education researchers and clinical educators, and to advance new knowledge regarding clinical education practices.
Official journal of the The Netherlands Association of Medical Education (NVMO).
Perspectives on Medical Education is a non-profit Open Access journal with no charges for authors to submit or publish an article, and the full text of all articles is freely available immediately upon publication, thanks to the sponsorship of The Netherlands Association for Medical Education.
Perspectives on Medical Education is highly visible thanks to its unrestricted online access policy.
Perspectives on Medical Education positions itself at the dynamic intersection of educational research and clinical education. While other journals in the health professional education domain orient predominantly to education researchers or to clinical educators, Perspectives positions itself at the collaborative interface between these perspectives. This unique positioning reflects the journal’s mission to support and enrich collaborative scholarship between education researchers and clinical educators, and to advance new knowledge regarding clinical education practices. Reflecting this mission, the journal both welcomes original research papers arising from scholarly collaborations among clinicians, teachers and researchers and papers providing resources to develop the community’s ability to conduct such collaborative research. The journal’s audience includes researchers and practitioners: researchers who wish to explore challenging questions of health professions education and clinical teachers who wish to both advance their practice and envision for themselves a collaborative role in scholarly educational innovation. This audience of researchers, clinicians and educators is both international and interdisciplinary.
The journal has a long history. In 1982, the journal was founded by the Dutch Association for Medical Education, as a Dutch language journal (Netherlands Journal of Medical Education). As a Dutch journal it fuelled educational research and innovation in the Netherlands. It is one of the factors for the Dutch success in medical education. In 2012, it widened its scope, transforming into an international English language journal. The journal swiftly became international in all aspects: the readers, authors, reviewers and editorial board members.
The editorial board members represent the different parental disciplines in the field of medical education, e.g. clinicians, social scientists, biomedical scientists, statisticians and linguists. Several of them are leading scholars. Three of the editors are in the top ten of most cited authors in the medical education field. Two editors were awarded the Karolinska Institute Prize for Research. Presently, Erik Driessen leads the journal as Editor in Chief.
Perspectives on Medical Education is highly visible thanks to its unrestricted online access policy. It is sponsored by theThe Netherlands Association of Medical Education and offers free manuscript submission.
Perspectives on Medical Education positions itself at the dynamic intersection of educational research and clinical education. While other journals in the health professional education domain orient predominantly to education researchers or to clinical educators, Perspectives positions itself at the collaborative interface between these perspectives. This unique positioning reflects the journal’s mission to support and enrich collaborative scholarship between education researchers and clinical educators, and to advance new knowledge regarding clinical education practices. Reflecting this mission, the journal both welcomes original research papers arising from scholarly collaborations among clinicians, teachers and researchers and papers providing resources to develop the community’s ability to conduct such collaborative research. The journal’s audience includes researchers and practitioners: researchers who wish to explore challenging questions of health professions education and clinical teachers who wish to both advance their practice and envision for themselves a collaborative role in scholarly educational innovation. This audience of researchers, clinicians and educators is both international and interdisciplinary.
The journal has a long history. In 1982, the journal was founded by the Dutch Association for Medical Education, as a Dutch language journal (Netherlands Journal of Medical Education). As a Dutch journal it fuelled educational research and innovation in the Netherlands. It is one of the factors for the Dutch success in medical education. In 2012, it widened its scope, transforming into an international English language journal. The journal swiftly became international in all aspects: the readers, authors, reviewers and editorial board members.
The editorial board members represent the different parental disciplines in the field of medical education, e.g. clinicians, social scientists, biomedical scientists, statisticians and linguists. Several of them are leading scholars. Three of the editors are in the top ten of most cited authors in the medical education field. Two editors were awarded the Karolinska Institute Prize for Research. Presently, Erik Driessen leads the journal as Editor in Chief.
Perspectives on Medical Education is highly visible thanks to its unrestricted online access policy. It is sponsored by theThe Netherlands Association of Medical Education and offers free manuscript submission.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
