International Medical Education最新文献

{"title":"Examining the Efficacy of ChatGPT in Marking Short-Answer Assessments in an Undergraduate Medical Program","authors":"Leo Morjaria, Levi Burns, Keyna Bracken, Anthony J Levinson, Quang N. Ngo, Mark Lee, Matthew Sibbald","doi":"10.3390/ime3010004","DOIUrl":"https://doi.org/10.3390/ime3010004","url":null,"abstract":"Traditional approaches to marking short-answer questions face limitations in timeliness, scalability, inter-rater reliability, and faculty time costs. Harnessing generative artificial intelligence (AI) to address some of these shortcomings is attractive. This study aims to validate the use of ChatGPT for evaluating short-answer assessments in an undergraduate medical program. Ten questions from the pre-clerkship medical curriculum were randomly chosen, and for each, six previously marked student answers were collected. These sixty answers were evaluated by ChatGPT in July 2023 under four conditions: with both a rubric and standard, with only a standard, with only a rubric, and with neither. ChatGPT displayed good Spearman correlations with a single human assessor (r = 0.6–0.7, p < 0.001) across all conditions, with the absence of a standard or rubric yielding the best correlation. Scoring differences were common (65–80%), but score adjustments of more than one point were less frequent (20–38%). Notably, the absence of a rubric resulted in systematically higher scores (p < 0.001, partial η2 = 0.33). Our findings demonstrate that ChatGPT is a viable, though imperfect, assistant to human assessment, performing comparably to a single expert assessor. This study serves as a foundation for future research on AI-based assessment techniques with potential for further optimization and increased reliability.","PeriodicalId":510221,"journal":{"name":"International Medical Education","volume":"34 8","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139611874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Analysis of Patient Information and Differential Diagnosis with Clinical Reasoning in Pre-Clinical Medical Students","authors":"Dichitchai Mettarikanon, Weeratian Tawanwongsri","doi":"10.3390/ime3010003","DOIUrl":"https://doi.org/10.3390/ime3010003","url":null,"abstract":"Background: The development of clinical reasoning (CR) abilities in Thai medical students during their pre-clinical years lacked well-designed establishment. Methods: This study utilized a pre-test and post-test design without a control group and was conducted at Walailak University, Thailand, in May 2022. We collected participant baseline characteristics and compared scores evaluated by the instructor and participants before and after the intervention. Additionally, we conducted a post-intervention survey on workshop satisfaction, perspectives on CR, and its learning impact. Results: Nineteen third-year medical students were included in the analysis, and twelve (63.2%) were women. The mean age was 20.6 years (standard deviation, SD: 0.5). The total score evaluated by the instructor after the intervention (8.95; SD, 1.81) was significantly higher than that before the intervention (1.68; SD, 0.67), p < 0.001. The total score evaluated by the participants after the intervention (8.22; SD = 1.44) was significantly higher than that before the intervention (2.34; SD, 2.06), p < 0.001. Overall, satisfaction was high. Conclusions: A short interactive workshop effectively taught CR principles and practice to Thai third-year medical students. These findings support the possibility of implementing this in pre-clinical curricula to prepare them for clinical practice.","PeriodicalId":510221,"journal":{"name":"International Medical Education","volume":"53 38","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139527652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generation of Medical Case-Based Multiple-Choice Questions","authors":"Somaiya Al Shuriaqi, Abdulrahman Aal Abdulsalam, Ken Masters","doi":"10.3390/ime3010002","DOIUrl":"https://doi.org/10.3390/ime3010002","url":null,"abstract":"This narrative review is a detailed look at how we make multiple-choice questions (MCQs) based on medical cases in today’s medical teaching. Moving from old-style MCQs to ones that are more related to real clinical situations is really important. It helps in growing critical thinking and practical use, especially since MCQs are still the primary method for testing knowledge in medicine. We look at the history, design ideas, and both manual and computer-based methods that have helped create MCQs. Technologies like Artificial Intelligence (AI) and Natural Language Processing (NLP) are receiving a lot of focus for their ability to automate the creation of question. We also talk about the challenges of using real patient cases, like the need for exact clinical information, reducing unclear information, and thinking about ethical issues. We also investigate the measures of validity and reliability that are crucial to maintaining the honesty of case-based MCQs. Finally, we look ahead, speculating on where medical education is headed as new technologies are incorporated and the value of case-based evaluations continues to rise.","PeriodicalId":510221,"journal":{"name":"International Medical Education","volume":"15 7","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139158287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Follow-Up of Post Myocardial Infarction Using Telemedicine: Stakeholders’ Education, Results and Customer Satisfaction","authors":"Manuela Bocchino, Francesco Gabbrielli, G. Pastena, N. Danisi, Fabio Ferranti, Raffaele Scardala, Mariagrazia Romano, Claudia Sorrentino, Fabrizio Ammirati","doi":"10.3390/ime3010001","DOIUrl":"https://doi.org/10.3390/ime3010001","url":null,"abstract":"Background. There are few studies about post myocardial infarction follow-up using telemedicine. We organized a post-discharge telemedicine service with a dedicated team. To do this, it was necessary that all stakeholders involved in the organization and use of the telemedicine service were properly educated and informed. Methods. We designed a theoretical–practical mini-course to train healthcare personnel and increase skills, with excellent learning outcomes and satisfaction. Thereafter, we enrolled patients affected by acute myocardial infarction with ST elevation (STEMI), MINOCA (myocardial infarction with no obstructive coronary atherosclerosis), Takotsubo syndrome or spontaneous coronary dissection, and high-risk acute myocardial infarction without ST elevation (NSTEMI). At discharge, the cardiology technician performed counselling for the patient, using regional platforms, such as televisit, at 1 and 4 months, allowed us to monitor major adverse cardiac events (MACE), heart failure, arrhythmias, unstable angina and non-cardiovascular events, therapy adherence, target therapy and customer satisfaction. Results. Between November 2021 and February 2023, we enrolled 110 patients: 72% affected by STEMI, 22% by NSTEMI. At the 1-month follow up, 12 patients did not reach the pressure target and 23 patients did not reach the LDL target. We observed three patients requiring hospital readmission, three requiring hospital visits for further investigation, and one death. To date, a four month follow up was performed for 54 patients. No readmissions or deaths occurred. We detected a rate of 96% of customer satisfaction. Conclusions. A health coordination center with a dedicated team makes televisit safe as a follow-up for post-myocardial infarction patients. Beforehand, it is fundamental for healthcare professionals to acquire theoretical knowledge and updates, and the acquisition of manual, technical and practical skills.","PeriodicalId":510221,"journal":{"name":"International Medical Education","volume":"5 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139167930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Stepping into a Preclinical Course as a Clinical Guest Lecturer","authors":"Kristen M. Coppola, Hanin Rashid, Sarang Kim, R. Lebeau","doi":"10.3390/ime2040026","DOIUrl":"https://doi.org/10.3390/ime2040026","url":null,"abstract":"Medical schools often rely on clinicians to provide guest lectures in foundational science courses. These guest lecturers may teach only once or sporadically in the undergraduate curriculum, which brings about several challenges that can impact their effectiveness. This guide, drawn from evidence-based teaching principles, medical student feedback, the authors’ training in educational and cognitive psychology, and experience as a clinical teacher, is designed to support clinical guest lecturers in meeting the educational standards and expectations for teaching in the preclinical setting as well as give insight into the needs of foundational science course directors and students.","PeriodicalId":510221,"journal":{"name":"International Medical Education","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139214661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}