Journal of Educational Evaluation for Health Professions最新文献

{"title":"Efficacy of a short multimodal workshop including stress management and communication skills training on objective structured clinical examination performance and exam-related anxiety in French medical students: a randomized controlled study.","authors":"Meryl Vedrenne-Cloquet, Etienne de Montmollin, Augustin Gaudemer, Marion Strullu, Noémie De Cacqueray, Guillaume Beinse, Philibert Duriez, Marie-Aude Piot, Alain Cariou, Donia Bouzid, Damien Roux","doi":"10.3352/jeehp.2026.23.19","DOIUrl":"10.3352/jeehp.2026.23.19","url":null,"abstract":"<p><strong>Purpose: </strong>To assess the effectiveness of a short multimodal workshop, including stress-management and communication-skills training, on objective structured clinical examination (OSCE) performance and exam-related anxiety among medical students identified as needing additional support.</p><p><strong>Methods: </strong>We conducted a prospective, open-label, randomized controlled trial including fifth-year medical students with the lowest scores on an initial mock OSCE. Participants were randomized 1:1 to the intervention group (stress management, communication training, and OSCE simulation) or the control group. OSCE examiners were blinded to group allocation. Anxiety was measured using the State-Trait Anxiety Inventory-State scale (STAI-State) before the intervention (T0), after the intervention (T1; intervention group only), and before the second mock OSCE (T2). OSCE scores and anxiety levels were compared between groups, and changes in anxiety over time were analyzed among students in the intervention group.</p><p><strong>Results: </strong>A total of 109 students were included in the per-protocol analysis (49 intervention, 60 control). Global OSCE scores at T2 did not differ between groups (median [interquartile range], 13.0/20 [11.9-14.3] vs. 13.3/20 [12.0-14.6]; between-group difference, 0.08 [95% confidence interval, -0.14 to 0.29]; P=0.50). Anxiety levels were moderate, with no significant between-group difference (STAI-State score, 47 [42-50], n=8, in the intervention group vs. 44 [42-46], n=12; P=0.32). A transient increase in situational anxiety was observed immediately after training in some intervention participants. No correlation was found between anxiety and OSCE performance.</p><p><strong>Conclusion: </strong>A short multimodal workshop focused on stress management and communication training did not improve OSCE performance or OSCE-related anxiety, suggesting that single-session interventions may be insufficient.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"23 ","pages":"19"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148278636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Practicing pediatric interviews with parents through conversational AI-based virtual patients: an observational study in Spanish undergraduate medical education.","authors":"César Fernández, Francisco Sánchez-Ferrer, María Asunción Vicente","doi":"10.3352/jeehp.2026.23.27","DOIUrl":"10.3352/jeehp.2026.23.27","url":null,"abstract":"<p><strong>Purpose: </strong>To evaluate students' acceptance of training via artificial intelligence (AI)-based virtual patients (VPs) in pediatrics; to analyze the main factors that affect student satisfaction; and to compare student satisfaction with that reported in the literature.</p><p><strong>Methods: </strong>Two observational studies were carried out. Study S1 analyzed students' interactions with the platform and study S2 analyzed their answers to a final questionnaire. All students enrolled in the \"Pediatrics II and Pediatric Surgery\" course were invited to participate during a continuous session on May 7, 2025. Study S1 analyzed whether the case/session order, the number of interactions, the time spent, and student gender were associated with student satisfaction (by fitting a linear mixed-effects model); and also performed a qualitative analysis of students' open-ended comments. Study S2 analyzed the influence of demographic data on students' feedback (by fitting proportional odds ordinal logistic regression models and linear models). All experimental data, code and results are available.</p><p><strong>Results: </strong>Concerning study S1, 70 students participated. Platform rating was high (mean=9.04, standard deviation=1.09) and was positively associated with the number of interactions with the VPs (P=0.038, β=0.022 [0.001-0.043]). Concerning study S2, 60 students participated. No statistically significant influence of demographics was found either for answers to individual questionnaire items or for grouped answers.</p><p><strong>Conclusion: </strong>Student acceptance of training with AI-based VPs was supported by the satisfaction outcomes measured and by comparison with previous studies. However, further research is needed to confirm these findings.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"23 ","pages":"27"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148867180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Accuracy of ChatGPT in answering cardiology board-style questions.","authors":"Albert Andrew","doi":"10.3352/jeehp.2025.22.9","DOIUrl":"10.3352/jeehp.2025.22.9","url":null,"abstract":"","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"9"},"PeriodicalIF":9.3,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12042102/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143517011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The impact of artificial intelligence-driven simulation on the development of non-technical skills in medical education: a systematic review.","authors":"Sana Loubbairi, Yasmine El Moussaoui, Laila Lahlou, Imad Chakri, Hicham Nassik","doi":"10.3352/jeehp.2025.22.37","DOIUrl":"https://doi.org/10.3352/jeehp.2025.22.37","url":null,"abstract":"<p><strong>Purpose: </strong>Artificial intelligence (AI)-driven simulation is an emerging approach in healthcare education that enhances learning effectiveness. This review examined its impact on the development of non-technical skills among medical learners.</p><p><strong>Methods: </strong>Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a systematic review was conducted using the following databases: Web of Science, ScienceDirect, Scopus, and PubMed. The quality of the included studies was assessed using the Mixed.</p><p><strong>Methods: </strong>Appraisal Tool. The protocol was previously registered in PROSPERO (CRD420251038024).</p><p><strong>Results: </strong>Of the 1,442 studies identified in the initial search, 20 met the inclusion criteria, involving 2,535 participants. The simulators varied considerably, ranging from platforms built on symbolic AI methods to social robots powered by computational AI. Among the 15 AI-driven simulators, 10 used ChatGPT or its variants as virtual patients. Several studies evaluated multiple non-technical skills simultaneously. Communication and clinical reasoning were the most frequently assessed skills, appearing in 12 and 6 studies, respectively, which generally reported positive outcomes. Improvements were also noted in decision-making, empathy, self-confidence, critical thinking, and problem-solving. In contrast, emotional regulation, assessed in a single study, showed no significant difference. Notably, none of the studies examined reflection, reflective practice, teamwork, or leadership.</p><p><strong>Conclusion: </strong>AI-driven simulation shows substantial potential for enhancing non-technical skills in medical education, particularly communication and clinical reasoning. However, its effects on several other non-technical skills remain unclear. Given heterogeneity in study designs and outcome measures, these findings should be interpreted cautiously. These considerations highlight the need for further research to support integrating this innovative approach into medical curricula.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"37"},"PeriodicalIF":3.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146020145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Performance of large language models on Thailand’s national medical licensing examination: a cross-sectional study.","authors":"Prut Saowaprut, Romen Samuel Wabina, Junwei Yang, Lertboon Siriwat","doi":"10.3352/jeehp.2025.22.16","DOIUrl":"10.3352/jeehp.2025.22.16","url":null,"abstract":"<p><strong>Purpose: </strong>This study aimed to evaluate the feasibility of general-purpose large language models (LLMs) in addressing inequities in medical licensure exam preparation for Thailand’s National Medical Licensing Examination (ThaiNLE), which currently lacks standardized public study materials.</p><p><strong>Methods: </strong>We assessed 4 multi-modal LLMs (GPT-4, Claude 3 Opus, Gemini 1.0/1.5 Pro) using a 304-question ThaiNLE Step 1 mock examination (10.2% image-based), applying deterministic API configurations and 5 inference repetitions per model. Performance was measured via micro- and macro-accuracy metrics compared against historical passing thresholds.</p><p><strong>Results: </strong>All models exceeded passing scores, with GPT-4 achieving the highest accuracy (88.9%; 95% confidence interval, 88.7–89.1), surpassing Thailand’s national average by more than 2 standard deviations. Claude 3.5 Sonnet (80.1%) and Gemini 1.5 Pro (72.8%) followed hierarchically. Models demonstrated robustness across 17 of 20 medical domains, but variability was noted in genetics (74.0%) and cardiovascular topics (58.3%). While models demonstrated proficiency with images (Gemini 1.0 Pro: +9.9% vs. text), text-only accuracy remained superior (GPT4o: 90.0% vs. 82.6%).</p><p><strong>Conclusion: </strong>General-purpose LLMs show promise as equitable preparatory tools for ThaiNLE Step 1. However, domain-specific knowledge gaps and inconsistent multi-modal integration warrant refinement before clinical deployment.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"16"},"PeriodicalIF":9.3,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143986836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The role of large language models in the peer-review process: opportunities and challenges for medical journal reviewers and editors.","authors":"Jisoo Lee, Jieun Lee, Jeong-Ju Yoo","doi":"10.3352/jeehp.2025.22.4","DOIUrl":"10.3352/jeehp.2025.22.4","url":null,"abstract":"<p><p>The peer review process ensures the integrity of scientific research. This is particularly important in the medical field, where research findings directly impact patient care. However, the rapid growth of publications has strained reviewers, causing delays and potential declines in quality. Generative artificial intelligence, especially large language models (LLMs) such as ChatGPT, may assist researchers with efficient, high-quality reviews. This review explores the integration of LLMs into peer review, highlighting their strengths in linguistic tasks and challenges in assessing scientific validity, particularly in clinical medicine. Key points for integration include initial screening, reviewer matching, feedback support, and language review. However, implementing LLMs for these purposes will necessitate addressing biases, privacy concerns, and data confidentiality. We recommend using LLMs as complementary tools under clear guidelines to support, not replace, human expertise in maintaining rigorous peer review standards.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"4"},"PeriodicalIF":9.3,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11952698/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143693856","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Effectiveness of interprofessional education enhanced by live consultation observations for healthcare students and new professionals in Singapore: a retrospective cross-sectional study.","authors":"Lynette Mei Lim Goh, Wai Leong Chiu, Sky Wei Chee Koh","doi":"10.3352/jeehp.2025.22.21","DOIUrl":"10.3352/jeehp.2025.22.21","url":null,"abstract":"<p><p>This study aims to evaluate whether incorporating live consultation observations into interprofessional education (IPE) improves learning evaluation scores among healthcare professionals and students. A retrospective cross-sectional analysis was conducted using evaluation data from AHP IPE sessions held from January 2020 to December 2023 across 7 primary care clinics in Singapore. Evaluation scores were compared between sessions with facilitated discussions only (n=667) and sessions with additional live consultation observations (n=501). Logistic regression was used to analyze factors associated with perfect evaluation scores. Sessions that included live consultations were significantly more likely to achieve perfect evaluation scores (odds ratio [OR], 1.68; 95% confidence interval [CI], 1.27-2.22). Nursing/care coordinator and allied health professions (OR 2.07 and 1.76 respectively) were significantly more likely to give perfect scores compared to medical professions. Healthcare professionals were also more likely to give perfect scores than students (OR, 1.52; 95% CI,1.08-2.14), indicating enhanced perceived effectiveness. These findings support the use of experiential learning strategies to optimize interprofessional training outcomes.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"21"},"PeriodicalIF":3.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12768549/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145534594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Development and psychometric assessment of a scale for evaluating healthcare professionals' attitudes toward interprofessional education and collaboration in the United States: a cross-sectional study.","authors":"Michael Christopher Banks, Ryan Brock Mutcheson, Maedot Ariaya Haymete, Serkan Toy","doi":"10.3352/jeehp.2025.22.32","DOIUrl":"10.3352/jeehp.2025.22.32","url":null,"abstract":"<p><strong>Purpose: </strong>Interprofessional education (IPE) is increasingly recognized as critical to preparing health professionals for collaborative practice, yet rigorous assessment remains limited by a lack of psychometrically sound instruments. Building on a previously developed questionnaire for physicians, this study aimed to expand the scale to include allied health professionals and to evaluate whether the factor structure remained consistent across professions. We hypothesized that a similar factor structure would emerge from the combined dataset, thereby supporting the scale's generalizability.</p><p><strong>Methods: </strong>This observational study included 930 healthcare professionals in the United States (379 physicians, 419 nurses, 76 pharmacists, and others) who completed a 35-item questionnaire addressing IPE competency domains. Data were collected between December 2019 and May 2020. Exploratory factor analysis was employed to examine the factor structure, followed by item response theory (IRT) analyses to assess item fit, reliability, and validity. Raw data are available upon request.</p><p><strong>Results: </strong>Factor analysis of 22 retained items confirmed a 5-factor solution: teamwork and communication, patient-centered care, roles and responsibilities, ethics and attitudes, and reflective practice, explaining 59% of the variance. Subscale reliabilities ranged from α=0.65 to 0.87. IRT analyses supported construct validity and measurement precision, while identifying areas for refinement in reflective practice.</p><p><strong>Conclusion: </strong>This study demonstrates that the scale is reliable, valid, and generalizable across diverse health professions. It provides a robust tool for assessing attitudes toward IPE, offering value for curriculum evaluation, institutional benchmarking, and future longitudinal research on professional identity formation and collaborative practice.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"32"},"PeriodicalIF":3.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12768546/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145330367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Radiotorax.es: a web-based tool for formative self-assessment in chest X-ray interpretation.","authors":"Verónica Illescas-Megías, Jorge Manuel Maqueda-Pérez, Dolores Domínguez-Pinos, Teodoro Rudolphi Solero, Francisco Sendra-Portero","doi":"10.3352/jeehp.2025.22.17","DOIUrl":"10.3352/jeehp.2025.22.17","url":null,"abstract":"<p><p>Radiotorax.es is a free, non-profit web-based tool designed to support formative self-assessment in chest X-ray interpretation. This article presents its structure, educational applications, and usage data from 11 years of continuous operation. Users complete interpretation rounds of 20 clinical cases, compare their reports with expert evaluations, and conduct a structured self-assessment. From 2011 to 2022, 14,389 users registered, and 7,726 completed at least one session. Most were medical students (75.8%), followed by residents (15.2%) and practicing physicians (9.0%). The platform has been integrated into undergraduate medical curricula and used in various educational contexts, including tutorials, peer and expert review, and longitudinal tracking. Its flexible design supports self-directed learning, instructor-guided use, and multicenter research. As a freely accessible resource based on real clinical cases, Radiotorax.es provides a scalable, realistic, and well-received training environment that promotes diagnostic skill development, reflection, and educational innovation in radiology education.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"17"},"PeriodicalIF":3.7,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144250213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A nationwide survey on the curriculum and educational resources related to the Clinical Skills Test of the Korean Medical Licensing Examination: a cross-sectional descriptive study.","authors":"Eun-Kyung Chung, Seok Hoon Kang, Do-Hoon Kim, MinJeong Kim, Ji-Hyun Seo, Keunmi Lee, Eui-Ryoung Han","doi":"10.3352/jeehp.2025.22.11","DOIUrl":"10.3352/jeehp.2025.22.11","url":null,"abstract":"<p><strong>Purpose: </strong>The revised Clinical Skills Test (CST) of the Korean Medical Licensing Exam aims to provide a better assessment of physicians’ clinical competence and ability to interact with patients. This study examined the impact of the revised CST on medical education curricula and resources nationwide, while also identifying areas for improvement within the revised CST.</p><p><strong>Methods: </strong>This study surveyed faculty responsible for clinical clerkships at 40 medical schools throughout Korea to evaluate the status and changes in clinical skills education, assessment, and resources related to the CST. The researchers distributed the survey via email through regional consortia between December 7, 2023 and January 19, 2024.</p><p><strong>Results: </strong>Nearly all schools implemented preliminary student–patient encounters during core clinical rotations. Schools primarily conducted clinical skills assessments in the third and fourth years, with a simplified form introduced in the first and second years. Remedial education was conducted through various methods, including oneon-one feedback from faculty after the assessment. All schools established clinical skills centers and made ongoing improvements. Faculty members did not perceive the CST revisions as significantly altering clinical clerkship or skills assessments. They suggested several improvements, including assessing patient records to improve accuracy and increasing the objectivity of standardized patient assessments to ensure fairness.</p><p><strong>Conclusion: </strong>During the CST, students’ involvement in patient encounters and clinical skills education increased, improving the assessment and feedback processes for clinical skills within the curriculum. To enhance students’ clinical competencies and readiness, strengthening the validity and reliability of the CST is essential.</p>","PeriodicalId":46098,"journal":{"name":"Journal of Educational Evaluation for Health Professions","volume":"22 ","pages":"11"},"PeriodicalIF":9.3,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12042100/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143617568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}