Medical Teacher最新文献

{"title":"Inter-school variations in the standard of examiners' graduation-level OSCE judgements.","authors":"Peter Yeates, Adriano Maluf, Gareth McCray, Ruth Kinston, Natalie Cope, Kathy Cullen, Vikki O'Neill, Aidan Cole, Ching-Wa Chung, Rhian Goodfellow, Rebecca Vallender, Sue Ensaff, Rikki Goddard-Fuller, Robert McKinley","doi":"10.1080/0142159X.2024.2372087","DOIUrl":"10.1080/0142159X.2024.2372087","url":null,"abstract":"<p><strong>Introduction: </strong>Ensuring equivalence in high-stakes performance exams is important for patient safety and candidate fairness. We compared inter-school examiner differences within a shared OSCE and resulting impact on students' pass/fail categorisation.</p><p><strong>Methods: </strong>The same 6 station formative OSCE ran asynchronously in 4 medical schools, with 2 parallel circuits/school. We compared examiners' judgements using Video-based Examiner Score Comparison and Adjustment (VESCA): examiners scored station-specific comparator videos in addition to 'live' student performances, enabling 1/controlled score comparisons by a/examiner-cohorts and b/schools and 2/data linkage to adjust for the influence of examiner-cohorts. We calculated score impact and change in pass/fail categorisation by school.</p><p><strong>Results: </strong>On controlled video-based comparisons, inter-school variations in examiners' scoring (16.3%) were nearly double within-school variations (8.8%). Students' scores received a median adjustment of 5.26% (IQR 2.87-7.17%). The impact of adjusting for examiner differences on students' pass/fail categorisation varied by school, with adjustment reducing failure rate from 39.13% to 8.70% (school 2) whilst increasing failure from 0.00% to 21.74% (school 4).</p><p><strong>Discussion: </strong>Whilst the formative context may partly account for differences, these findings query whether variations may exist between medical schools in examiners' judgements. This may benefit from systematic appraisal to safeguard equivalence. VESCA provided a viable method for comparisons.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"735-743"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141559124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Unveiling the paradoxes of implementing post graduate competency based medical education programs.","authors":"Noa A Birman, Dana R Vashdi, Rotem Miller-Mor Atias, Arieh Riskin, Shmuel Zangen, Ita Litmanovitz, Doron Sagi","doi":"10.1080/0142159X.2024.2356826","DOIUrl":"10.1080/0142159X.2024.2356826","url":null,"abstract":"<p><strong>Purpose: </strong>Competency-based medical education (CBME) has gained prominence as an innovative model for post-graduate medical education, yet its implementation poses significant challenges, especially with regard to its sustainability. Drawing on paradox theory, we suggest that revealing the paradoxes underlying these challenges may contribute to our understanding of post graduate competency-based medical education (PGCBME) implementation processes and serve as a first-step in enhancing better implementation. Thus, the purpose of the current study is to identify the paradoxes associated with PGCBME implementation.</p><p><strong>Method: </strong>A qualitative study was conducted, as part of a larger action research, using in-depth semi-structured interviews with fellows and educators in eight Neonatal wards.</p><p><strong>Results: </strong>Analysis revealed that the PGCBME program examined in this study involves three different levels of standardization, each serving as one side of paradoxical tensions; (1) a paradox between the need for standardized assessment tools and for free-flow flexible assessment tools, (2) a paradox between the need for a standardized implementation process across all wards and the need for unique implementation protocols in each ward; and 3) a paradox between the need for a standardized meaning of competency proficiency and the need for flexible and personal competency achievement indicators.</p><p><strong>Conclusions: </strong>Implementing PGCBME programs involves many challenges, some of which are paradoxical, i.e. two contradictory challenges in which solving one challenge exacerbates another. Revealing these paradoxes is important in navigating them successfully.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"622-629"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141157988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Model for an Academic Medicine Interest Group: A student-led initiative.","authors":"Dawn M Szeltner, Timothy C Frommeyer, Garrett V Brittain, Michael M Gilbert, Brian J Schutter, Courtney Johnson-Gonzalez, Mark P Ortenzio, David P Muwanga, Reid M Fursmidt, Yanil V Ramirez, Nickolas A Stewart, Dean X Parmelee, Priti P Parikh","doi":"10.1080/0142159X.2024.2359956","DOIUrl":"10.1080/0142159X.2024.2359956","url":null,"abstract":"<p><p>Academic physicians are responsible for the education of medical students, residents, and other practicing physicians through clinical rotations lectures, seminars, research, and conferences. Therefore, the increasing need to recruit academic physicians holds immense value within the healthcare system. Academic Medicine Interest Group (AMIG) is a collective made up of students who share an interest in the growth and advancement of academic medicine. We present a guide and model on establishing an AMIG. We found that AMIG fostered professional growth by providing leadership, research, and teaching opportunities. Strategic planning, effective leadership, and group organization were all necessary for the success of the group.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"660-664"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141419830","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"ChatGPT as an item calibration tool: Psychometric insights in a high-stakes examination.","authors":"Daniela S M Pereira, Francisco Mourão, João Carlos Ribeiro, Patrício Costa, Serafim Guimarães, José Miguel Pêgo","doi":"10.1080/0142159X.2024.2376205","DOIUrl":"10.1080/0142159X.2024.2376205","url":null,"abstract":"<p><strong>Introduction: </strong>ChatGPT has attracted a lot of interest worldwide for its versatility in a range of natural language tasks, including in the education and evaluation industry. It can automate time- and labor-intensive tasks with clear economic and efficiency gains.</p><p><strong>Methods: </strong>This study evaluated the potential of ChatGPT to automate psychometric analysis of test questions from the 2020 Portuguese National Residency Selection Exam (PNA). ChatGPT was queried 100 times with the 150 MCQ from the exam. Using ChatGPT's responses, difficulty indices were calculated for each question based on the proportion of correct answers. The predicted difficulty levels were compared to the actual difficulty levels of the 2020 exam MCQ's using methods from classical test theory.</p><p><strong>Results: </strong>ChatGPT's predicted item difficulty indices positively correlated with the actual item difficulties (r (148) = -0.372, <i>p</i> < .001), suggesting a general consistency between the real and the predicted values. There was also a moderate significant negative correlation between the difficulty index predicted by ChatGPT and the number of challenges (r (148) = -0.302, <i>p</i> < .001), highlighting ChatGPT's potential for identifying less problematic questions.</p><p><strong>Conclusion: </strong>These findings unveiled ChatGPT's potential as a tool for assessment development, proving its capability to predict the psychometric characteristics of high-stakes test items in automated item calibration without pre-testing in real-life scenarios.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"677-683"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141620435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Keeping humans in the loop efficiently by generating question templates instead of questions using AI: Validity evidence on Hybrid AIG.","authors":"Yavuz Selim Kıyak, Emre Emekli, Özlem Coşkun, Işıl İrem Budakoğlu","doi":"10.1080/0142159X.2024.2430360","DOIUrl":"10.1080/0142159X.2024.2430360","url":null,"abstract":"<p><strong>Background: </strong>Manually creating multiple-choice questions (MCQ) is inefficient. Automatic item generation (AIG) offers a scalable solution, with two main approaches: template-based and non-template-based (AI-driven). Template-based AIG ensures accuracy but requires significant expert input to develop templates. In contrast, AI-driven AIG can generate questions quickly but with inaccuracies. The Hybrid AIG combines the strengths of both methods. However, neither have MCQs been generated using the Hybrid AIG approach nor has any validity evidence been provided.</p><p><strong>Methods: </strong>We generated MCQs using the Hybrid AIG approach and investigated the validity evidence of these questions by determining whether experts could identify the correct answers. We used a custom ChatGPT to develop an item template, which were then fed into Gazitor, a template-based AIG (non-AI) software. A panel of medical doctors identified the answers.</p><p><strong>Results: </strong>Of 105 decisions, 101 (96.2%) matched the software's correct answer. In all MCQs (100%), the experts reached a consensus on the correct answer. The evidence corresponds to the 'Relations to Other Variables' in Messick's validity framework.</p><p><strong>Conclusions: </strong>The Hybrid AIG approach can enhance the efficiency of MCQ generation while maintaining accuracy. It mitigates concerns about hallucinations while benefiting from AI.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"744-747"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142739707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Mental health and wellness of medical students in Ontario - a mixed methods approach.","authors":"Usama Qureshi, Yusuf Qureshi, Armaan Khan, Adam Suleman","doi":"10.1080/0142159X.2024.2421992","DOIUrl":"10.1080/0142159X.2024.2421992","url":null,"abstract":"","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"756"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142564275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Concerns regarding the assessment of clinical reasoning in medical students.","authors":"Supianto","doi":"10.1080/0142159X.2024.2421256","DOIUrl":"10.1080/0142159X.2024.2421256","url":null,"abstract":"","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"755-756"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142504024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"ASPIRE for excellence in student engagement: Examples of how institutions operationalize a complex construct.","authors":"Barbara Barzansky, Carmen Fuentealba","doi":"10.1080/0142159X.2024.2368565","DOIUrl":"10.1080/0142159X.2024.2368565","url":null,"abstract":"<p><p>Student engagement is a key contributor to educational programme quality. It is a complex construct often defined with the focus on student behaviors. However, a broader, more organizational approach takes into account the institutional context and how this can be structured to encourage and support students' willingness and ability to become engaged. This paper includes suggestions for a student engagement system using key elements recognized in the literature and concrete examples from medical schools that achieved recognition in the ASPIRE-to-Excellence student engagement initiative. The examples from the ASPIRE participants demonstrate that the presence of certain key features creates a mutually beneficial collaborative approach between students and their institutions. This includes opportunities for students to formally engage in four domains touching on the breadth of student life, including institutional governance and policy-making, educational programme development and evaluation, participation in activities in the academic community, and participation in local community and international outreach. Based on an in-depth review of the information from three medical schools that recently received an ASPIRE-to-Excellence award in student engagement, it was possible to identify certain specific practices that individually and taken together allow an institution to demonstrate excellence in this complex construct. As an overarching concept, it was clear that student participation in each of these domain areas benefitted from a supportive institutional culture characterized by specific formal attributes and activities. Examples included codifying student involvement in governance through institutional policies; maximizing communication routes among students and between students and school administrators and faculty; and formalizing a participatory environment through missions statements or strategic plans. For programme planning, a helpful conceptualization is that a successful student engagement programme occurs when change is championed by all stakeholders within an institution and the organization supports a collaborative culture that includes students as active participants and partners.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"617-621"},"PeriodicalIF":3.3,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141427201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating the value of AI-generated questions for USMLE step 1 preparation: A study using ChatGPT-3.5.","authors":"Alan Balu, Stefan T Prvulovic, Claudia Fernandez Perez, Alexander Kim, Daniel A Donoho, Gregory Keating","doi":"10.1080/0142159X.2025.2478872","DOIUrl":"https://doi.org/10.1080/0142159X.2025.2478872","url":null,"abstract":"<p><strong>Purpose: </strong>Students are increasingly relying on artificial intelligence (AI) for medical education and exam preparation. However, the factual accuracy and content distribution of AI-generated exam questions for self-assessment have not been systematically investigated.</p><p><strong>Methods: </strong>Curated prompts were created to generate multiple-choice questions matching the USMLE Step 1 examination style. We utilized ChatGPT-3.5 to generate 50 questions and answers based upon each prompt style. We manually examined output for factual accuracy, Bloom's Taxonomy, and category within the USMLE Step 1 content outline.</p><p><strong>Results: </strong>ChatGPT-3.5 generated 150 multiple-choice case-style questions and selected an answer. Overall, 83% of generated multiple questions had no factual inaccuracies and 15% contained one to two factual inaccuracies. With simple prompting, common themes included deep venous thrombosis, myocardial infarction, and thyroid disease. Topic diversity improved by separating content topic generation from question generation, and specificity to Step 1 increased by indicating that \"treatment\" questions were not desired.</p><p><strong>Conclusion: </strong>We demonstrate that ChatGPT-3.5 can successfully generate Step 1 style questions with reasonable factual accuracy, and this method may be used by medical students preparing for USMLE examinations. While AI-generated questions demonstrated adequate factual accuracy, targeted prompting techniques should be used to overcome ChatGPT's bias towards particular medical conditions.</p>","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"1-9"},"PeriodicalIF":3.3,"publicationDate":"2025-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143730609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The role of internationalization of medical education via international exchange programs in professional identity formation.","authors":"Jason Luong, Jessica Hui, Geoffroy Noel, Anette Wu","doi":"10.1080/0142159X.2025.2478873","DOIUrl":"https://doi.org/10.1080/0142159X.2025.2478873","url":null,"abstract":"","PeriodicalId":18643,"journal":{"name":"Medical Teacher","volume":" ","pages":"1"},"PeriodicalIF":3.3,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143657844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}