重新设计的 "可委托专业活动（EPA）评估 "减少了体验环境中的分数膨胀。

IF 3.8 4区教育学 Q1 EDUCATION, SCIENTIFIC DISCIPLINES

American Journal of Pharmaceutical Education Pub Date : 2024-09-26 DOI:10.1016/j.ajpe.2024.101297

Kathryn Fuller, Nicole R. Pinelli, Adam M. Persky

{"title":"重新设计的 \"可委托专业活动（EPA）评估 \"减少了体验环境中的分数膨胀。","authors":"Kathryn Fuller, Nicole R. Pinelli, Adam M. Persky","doi":"10.1016/j.ajpe.2024.101297","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><div>This study aims to evaluate the impact of redesigning an entrustable professional activities (EPAs) assessment tool on the accuracy of student performance assessment within pharmacy education.</div></div><div><h3>Methods</h3><div>The study used retrospective programmatic data for students on clinical rotations over a 3-year period and compared entrustment levels assigned by preceptors with suggested entrustment levels. This tool was redesigned to separate formative EPA feedback from final grade determination. Data were analyzed using chi-squared tests to identify trends in students ABOVE, AT, or BELOW the suggested entrustment levels. Additionally, to account for intercohort variability, the relationship between students ABOVE the suggested level of entrustment and postgraduate metrics was examined.</div></div><div><h3>Results</h3><div>After the implementation of the revised tool, there was a significant decrease (−3%) in the percentage of students scoring ABOVE the suggested entrustment levels and an increase in the percentage of students scoring AT (+1%) or BELOW (+2%) the suggested entrustment levels. Changes were also observed in individual patient care settings, with a decrease in grade inflation and an increase in accurate assessments. North American Pharmacist Licensure Examination (NAPLEX) pass rates, residency match rates, and grade point average did not correlate with entrustment levels.</div></div><div><h3>Conclusion</h3><div>The redesigned EPA assessment tool demonstrated a decrease in grade inflation resulting in more accurate assessments. The tool’s focus on holistic grading and narrative descriptors contributed to better alignment between preceptor assessment and school-suggested achievement levels. This study suggests that EPA assessments in pharmacy education could benefit from a stronger emphasis on formative feedback and the use of holistic assessment methods for final grade determinations. The findings underscore the potential advantages of considering a separation between EPA scoring and final grades, prompting the Academy to explore their assessment practices to better reflect student performance in clinical experiences.</div></div>","PeriodicalId":55530,"journal":{"name":"American Journal of Pharmaceutical Education","volume":"88 11","pages":"Article 101297"},"PeriodicalIF":3.8000,"publicationDate":"2024-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Redesigned Entrustable Professional Activity (EPA) Assessments Reduce Grade Inflation in the Experiential Setting\",\"authors\":\"Kathryn Fuller, Nicole R. Pinelli, Adam M. Persky\",\"doi\":\"10.1016/j.ajpe.2024.101297\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Objective</h3><div>This study aims to evaluate the impact of redesigning an entrustable professional activities (EPAs) assessment tool on the accuracy of student performance assessment within pharmacy education.</div></div><div><h3>Methods</h3><div>The study used retrospective programmatic data for students on clinical rotations over a 3-year period and compared entrustment levels assigned by preceptors with suggested entrustment levels. This tool was redesigned to separate formative EPA feedback from final grade determination. Data were analyzed using chi-squared tests to identify trends in students ABOVE, AT, or BELOW the suggested entrustment levels. Additionally, to account for intercohort variability, the relationship between students ABOVE the suggested level of entrustment and postgraduate metrics was examined.</div></div><div><h3>Results</h3><div>After the implementation of the revised tool, there was a significant decrease (−3%) in the percentage of students scoring ABOVE the suggested entrustment levels and an increase in the percentage of students scoring AT (+1%) or BELOW (+2%) the suggested entrustment levels. Changes were also observed in individual patient care settings, with a decrease in grade inflation and an increase in accurate assessments. North American Pharmacist Licensure Examination (NAPLEX) pass rates, residency match rates, and grade point average did not correlate with entrustment levels.</div></div><div><h3>Conclusion</h3><div>The redesigned EPA assessment tool demonstrated a decrease in grade inflation resulting in more accurate assessments. The tool’s focus on holistic grading and narrative descriptors contributed to better alignment between preceptor assessment and school-suggested achievement levels. This study suggests that EPA assessments in pharmacy education could benefit from a stronger emphasis on formative feedback and the use of holistic assessment methods for final grade determinations. The findings underscore the potential advantages of considering a separation between EPA scoring and final grades, prompting the Academy to explore their assessment practices to better reflect student performance in clinical experiences.</div></div>\",\"PeriodicalId\":55530,\"journal\":{\"name\":\"American Journal of Pharmaceutical Education\",\"volume\":\"88 11\",\"pages\":\"Article 101297\"},\"PeriodicalIF\":3.8000,\"publicationDate\":\"2024-09-26\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"American Journal of Pharmaceutical Education\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0002945924110169\",\"RegionNum\":4,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION, SCIENTIFIC DISCIPLINES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Journal of Pharmaceutical Education","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0002945924110169","RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}

引用次数: 0

摘要

目的本研究旨在评估重新设计可委托专业活动（EPA）评估工具对药学教育中学生绩效评估准确性的影响：该研究利用三年内临床轮转学生的回顾性课程数据，比较了戒酒师分配的委托水平和建议的委托水平。该工具经过重新设计，将形成性 EPA 反馈与最终成绩评定分开。数据通过 Chi-Squared 检验进行分析，以寻找超出、达到或低于建议委托水平的学生趋势。此外，为了考虑同组学生之间的差异，还研究了超出建议委托水平的学生与毕业后指标之间的关系：结果：实施修订后的工具后，得分高于建议委托水平的学生明显减少（3%），而得分达到（1%）或低于（2%）的学生有所增加。个别病人护理环境也发生了变化，分数虚高现象减少，准确评估增加。NAPLEX 通过率、住院医生匹配率和平均学分绩点与委托水平没有关联：讨论：重新设计的 EPA 评估工具减少了分数膨胀，从而提高了评估的准确性。该工具注重整体评分和叙述性描述，有助于更好地协调实习医生评估和学校建议的成绩水平。这项研究表明，药学教育中的 EPA 评估可以从更加重视形成性反馈和使用整体评估方法来确定最终成绩中获益。研究结果强调了考虑将 EPA 评分与最终成绩分开的潜在优势，促使学院探索其评估实践，以更好地反映学生在临床经验中的表现。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Redesigned Entrustable Professional Activity (EPA) Assessments Reduce Grade Inflation in the Experiential Setting

Objective

This study aims to evaluate the impact of redesigning an entrustable professional activities (EPAs) assessment tool on the accuracy of student performance assessment within pharmacy education.

Methods

The study used retrospective programmatic data for students on clinical rotations over a 3-year period and compared entrustment levels assigned by preceptors with suggested entrustment levels. This tool was redesigned to separate formative EPA feedback from final grade determination. Data were analyzed using chi-squared tests to identify trends in students ABOVE, AT, or BELOW the suggested entrustment levels. Additionally, to account for intercohort variability, the relationship between students ABOVE the suggested level of entrustment and postgraduate metrics was examined.

Results

After the implementation of the revised tool, there was a significant decrease (−3%) in the percentage of students scoring ABOVE the suggested entrustment levels and an increase in the percentage of students scoring AT (+1%) or BELOW (+2%) the suggested entrustment levels. Changes were also observed in individual patient care settings, with a decrease in grade inflation and an increase in accurate assessments. North American Pharmacist Licensure Examination (NAPLEX) pass rates, residency match rates, and grade point average did not correlate with entrustment levels.

Conclusion

The redesigned EPA assessment tool demonstrated a decrease in grade inflation resulting in more accurate assessments. The tool’s focus on holistic grading and narrative descriptors contributed to better alignment between preceptor assessment and school-suggested achievement levels. This study suggests that EPA assessments in pharmacy education could benefit from a stronger emphasis on formative feedback and the use of holistic assessment methods for final grade determinations. The findings underscore the potential advantages of considering a separation between EPA scoring and final grades, prompting the Academy to explore their assessment practices to better reflect student performance in clinical experiences.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

American Journal of Pharmaceutical Education 医学-学科教育

CiteScore

4.30

自引率

15.20%

发文量

114

期刊介绍： The Journal accepts unsolicited manuscripts that have not been published and are not under consideration for publication elsewhere. The Journal only considers material related to pharmaceutical education for publication. Authors must prepare manuscripts to conform to the Journal style (Author Instructions). All manuscripts are subject to peer review and approval by the editor prior to acceptance for publication. Reviewers are assigned by the editor with the advice of the editorial board as needed. Manuscripts are submitted and processed online (Submit a Manuscript) using Editorial Manager, an online manuscript tracking system that facilitates communication between the editorial office, editor, associate editors, reviewers, and authors. After a manuscript is accepted, it is scheduled for publication in an upcoming issue of the Journal. All manuscripts are formatted and copyedited, and returned to the author for review and approval of the changes. Approximately 2 weeks prior to publication, the author receives an electronic proof of the article for final review and approval. Authors are not assessed page charges for publication.