ENTRUST 评估平台上的表现与能力本位外科教育中其他变量的相关性。

IF 2.6 3区医学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

Journal of Surgical Education Pub Date : 2024-11-11 DOI:10.1016/j.jsurg.2024.09.012

Lye-Yeng Wong MD , Nathan Lam BS , Yewon Ashley Son BS , Hyrum Eddington BS , Katherine D. Arnow MS , Jason Tsai MS , Ananya Anand MD , Fatyma C. Peralta MS , Samuel Shields MS , Edward F. Melcer PhD , Dana T. Lin MD, FACS , Cara A. Liebert MD, FACS

{"title":"ENTRUST 评估平台上的表现与能力本位外科教育中其他变量的相关性。","authors":"Lye-Yeng Wong MD , Nathan Lam BS , Yewon Ashley Son BS , Hyrum Eddington BS , Katherine D. Arnow MS , Jason Tsai MS , Ananya Anand MD , Fatyma C. Peralta MS , Samuel Shields MS , Edward F. Melcer PhD , Dana T. Lin MD, FACS , Cara A. Liebert MD, FACS","doi":"10.1016/j.jsurg.2024.09.012","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><div>With the implementation of American Board of Surgery (ABS) Entrustable Professional Activities (EPAs), there is continued need for objective, evidence-based assessment tools to augment existing microassessments and inform readiness for entrustment. The ENTRUST Assessment Platform is an online virtual-patient simulation platform to assess trainees’ surgical decision-making competence across preoperative, intraoperative, and postoperative phases of care. This study collects additional validity evidence for the ENTRUST platform in its relationship to other established variables in competency-based surgical education.</div></div><div><h3>Design</h3><div>This is a prospective analysis of surgical resident performance on the ENTRUST Right Lower Quadrant (RLQ) pain/Appendicitis EPA Assessment. ENTRUST scores were analyzed by PGY-level and correlations with Accreditation Council for Graduate Medical Education (ACGME) Case Logs, ACGME Surgery Milestones, and ABS In-Service Training Examination (ABSITE) scores were evaluated. Bivariate analyses were performed using Spearman rank correlations.</div></div><div><h3>Setting</h3><div>This study was conducted at a tertiary academic center (Stanford University, Palo Alto, CA) in a proctored exam setting.</div></div><div><h3>Participants</h3><div>Thirty-two PGY-1 though PGY-5 general surgery residents completed the ENTRUST RLQ Pain/Appendicitis EPA Assessment containing four case scenarios which were iteratively developed and scored by expert consensus and aligned with ABS EPA definitions.</div></div><div><h3>Results</h3><div>ENTRUST grand total score was positively correlated with PGY-level (rho = 0.57, p = 0.001), ACGME appendectomy case log volume (rho = 0.55, p = 0.002), and ABSITE raw score (rho = 0.66, p = 0.0004). ENTRUST performance was significantly correlated with all eighteen ACGME Surgery Milestones (rho = 0.43 to rho = 0.54, all p≤0.01), with the strongest correlation seen for PC1 (Patient Evaluation and Decision Making) (rho = 0.54, p = 0.006).</div></div><div><h3>Conclusions</h3><div>Performance on ENTRUST was significantly correlated with established variables in surgical training, including ACGME Appendectomy Case Logs, ABSITE, and ACGME Surgery Milestones. This study strengthens existing validity evidence for the ENTRUST Assessment Platform as an objective assessment of clinical decision-making. ENTRUST is an assessment tool which can augment microassessments and support competency-based medical education.</div></div>","PeriodicalId":50033,"journal":{"name":"Journal of Surgical Education","volume":"82 1","pages":"Article 103293"},"PeriodicalIF":2.6000,"publicationDate":"2024-11-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Correlation of Performance on the ENTRUST Assessment Platform With Other Variables in Competency-Based Surgical Education\",\"authors\":\"Lye-Yeng Wong MD , Nathan Lam BS , Yewon Ashley Son BS , Hyrum Eddington BS , Katherine D. Arnow MS , Jason Tsai MS , Ananya Anand MD , Fatyma C. Peralta MS , Samuel Shields MS , Edward F. Melcer PhD , Dana T. Lin MD, FACS , Cara A. Liebert MD, FACS\",\"doi\":\"10.1016/j.jsurg.2024.09.012\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Objective</h3><div>With the implementation of American Board of Surgery (ABS) Entrustable Professional Activities (EPAs), there is continued need for objective, evidence-based assessment tools to augment existing microassessments and inform readiness for entrustment. The ENTRUST Assessment Platform is an online virtual-patient simulation platform to assess trainees’ surgical decision-making competence across preoperative, intraoperative, and postoperative phases of care. This study collects additional validity evidence for the ENTRUST platform in its relationship to other established variables in competency-based surgical education.</div></div><div><h3>Design</h3><div>This is a prospective analysis of surgical resident performance on the ENTRUST Right Lower Quadrant (RLQ) pain/Appendicitis EPA Assessment. ENTRUST scores were analyzed by PGY-level and correlations with Accreditation Council for Graduate Medical Education (ACGME) Case Logs, ACGME Surgery Milestones, and ABS In-Service Training Examination (ABSITE) scores were evaluated. Bivariate analyses were performed using Spearman rank correlations.</div></div><div><h3>Setting</h3><div>This study was conducted at a tertiary academic center (Stanford University, Palo Alto, CA) in a proctored exam setting.</div></div><div><h3>Participants</h3><div>Thirty-two PGY-1 though PGY-5 general surgery residents completed the ENTRUST RLQ Pain/Appendicitis EPA Assessment containing four case scenarios which were iteratively developed and scored by expert consensus and aligned with ABS EPA definitions.</div></div><div><h3>Results</h3><div>ENTRUST grand total score was positively correlated with PGY-level (rho = 0.57, p = 0.001), ACGME appendectomy case log volume (rho = 0.55, p = 0.002), and ABSITE raw score (rho = 0.66, p = 0.0004). ENTRUST performance was significantly correlated with all eighteen ACGME Surgery Milestones (rho = 0.43 to rho = 0.54, all p≤0.01), with the strongest correlation seen for PC1 (Patient Evaluation and Decision Making) (rho = 0.54, p = 0.006).</div></div><div><h3>Conclusions</h3><div>Performance on ENTRUST was significantly correlated with established variables in surgical training, including ACGME Appendectomy Case Logs, ABSITE, and ACGME Surgery Milestones. This study strengthens existing validity evidence for the ENTRUST Assessment Platform as an objective assessment of clinical decision-making. ENTRUST is an assessment tool which can augment microassessments and support competency-based medical education.</div></div>\",\"PeriodicalId\":50033,\"journal\":{\"name\":\"Journal of Surgical Education\",\"volume\":\"82 1\",\"pages\":\"Article 103293\"},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2024-11-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Surgical Education\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S193172042400432X\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION, SCIENTIFIC DISCIPLINES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Surgical Education","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S193172042400432X","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

摘要

目的：随着美国外科学委员会（ABS）委托专业活动（EPAs）的实施，人们不断需要客观、循证的评估工具来增强现有的微观评估，并为委托准备工作提供信息。ENTRUST 评估平台是一个在线虚拟患者模拟平台，用于评估受训者在术前、术中和术后护理阶段的手术决策能力。本研究收集了 ENTRUST 平台与基于能力的外科教育中其他既定变量之间关系的更多有效性证据：本研究对外科住院医师在 ENTRUST 右下象限（RLQ）疼痛/阑尾炎 EPA 评估中的表现进行了前瞻性分析。按 PGY 级别对 ENTRUST 分数进行了分析，并评估了与毕业后医学教育认证委员会 (ACGME) 病例日志、ACGME 手术里程碑和 ABS 在职培训考试 (ABSITE) 分数之间的相关性。使用斯皮尔曼等级相关性进行双变量分析：本研究在一家三级学术中心（加利福尼亚州帕洛阿尔托斯坦福大学）进行，采用监考方式：32名PGY-1至PGY-5普外科住院医师完成了ENTRUST RLQ疼痛/阑尾炎EPA评估，该评估包含四种病例情景，由专家一致同意并根据ABS EPA定义反复开发和评分：ENTRUST总分与PGY级别（rho = 0.57，p = 0.001）、ACGME阑尾切除术病例记录量（rho = 0.55，p = 0.002）和ABSITE原始得分（rho = 0.66，p = 0.0004）呈正相关。ENTRUST成绩与ACGME所有18个手术里程碑均有明显相关性（rho = 0.43至rho = 0.54，p均小于0.01），其中PC1（患者评估与决策制定）的相关性最强（rho = 0.54，p = 0.006）：ENTRUST的成绩与外科培训中的既定变量（包括ACGME阑尾切除术病例日志、ABSITE和ACGME外科里程碑）明显相关。这项研究加强了 ENTRUST 评估平台作为临床决策客观评估的现有有效性证据。ENTRUST 是一种评估工具，可增强微观评估并支持基于能力的医学教育。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Correlation of Performance on the ENTRUST Assessment Platform With Other Variables in Competency-Based Surgical Education

Objective

With the implementation of American Board of Surgery (ABS) Entrustable Professional Activities (EPAs), there is continued need for objective, evidence-based assessment tools to augment existing microassessments and inform readiness for entrustment. The ENTRUST Assessment Platform is an online virtual-patient simulation platform to assess trainees’ surgical decision-making competence across preoperative, intraoperative, and postoperative phases of care. This study collects additional validity evidence for the ENTRUST platform in its relationship to other established variables in competency-based surgical education.

Design

This is a prospective analysis of surgical resident performance on the ENTRUST Right Lower Quadrant (RLQ) pain/Appendicitis EPA Assessment. ENTRUST scores were analyzed by PGY-level and correlations with Accreditation Council for Graduate Medical Education (ACGME) Case Logs, ACGME Surgery Milestones, and ABS In-Service Training Examination (ABSITE) scores were evaluated. Bivariate analyses were performed using Spearman rank correlations.

Setting

This study was conducted at a tertiary academic center (Stanford University, Palo Alto, CA) in a proctored exam setting.

Participants

Thirty-two PGY-1 though PGY-5 general surgery residents completed the ENTRUST RLQ Pain/Appendicitis EPA Assessment containing four case scenarios which were iteratively developed and scored by expert consensus and aligned with ABS EPA definitions.

Results

ENTRUST grand total score was positively correlated with PGY-level (rho = 0.57, p = 0.001), ACGME appendectomy case log volume (rho = 0.55, p = 0.002), and ABSITE raw score (rho = 0.66, p = 0.0004). ENTRUST performance was significantly correlated with all eighteen ACGME Surgery Milestones (rho = 0.43 to rho = 0.54, all p≤0.01), with the strongest correlation seen for PC1 (Patient Evaluation and Decision Making) (rho = 0.54, p = 0.006).

Conclusions

Performance on ENTRUST was significantly correlated with established variables in surgical training, including ACGME Appendectomy Case Logs, ABSITE, and ACGME Surgery Milestones. This study strengthens existing validity evidence for the ENTRUST Assessment Platform as an objective assessment of clinical decision-making. ENTRUST is an assessment tool which can augment microassessments and support competency-based medical education.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Journal of Surgical Education EDUCATION, SCIENTIFIC DISCIPLINES-SURGERY

CiteScore

5.60

自引率

10.30%

发文量

261

审稿时长

48 days

期刊介绍： The Journal of Surgical Education (JSE) is dedicated to advancing the field of surgical education through original research. The journal publishes research articles in all surgical disciplines on topics relative to the education of surgical students, residents, and fellows, as well as practicing surgeons. Our readers look to JSE for timely, innovative research findings from the international surgical education community. As the official journal of the Association of Program Directors in Surgery (APDS), JSE publishes the proceedings of the annual APDS meeting held during Surgery Education Week.