中国麻醉科住院医师评估里程碑:可靠性、有效性以及与客观考试分数的相关性:一项横断面研究。

Xia Ruan,Xiaohan Xu,Lijian Pei,Jie Yi,Chunhua Yu,Xuerong Yu,Bo Zhu,Xiang Quan,Xu Li,Hui Jv,Yuelun Zhang,Yuguang Huang
{"title":"中国麻醉科住院医师评估里程碑:可靠性、有效性以及与客观考试分数的相关性:一项横断面研究。","authors":"Xia Ruan,Xiaohan Xu,Lijian Pei,Jie Yi,Chunhua Yu,Xuerong Yu,Bo Zhu,Xiang Quan,Xu Li,Hui Jv,Yuelun Zhang,Yuguang Huang","doi":"10.1213/ane.0000000000007279","DOIUrl":null,"url":null,"abstract":"BACKGROUND\r\nEvaluating competency acquisition during residency training is crucial. The Anesthesiology Milestones have been implemented in the United States. The China Consortium of Elite Teaching Hospitals for Residency Education has also developed the Chinese Resident Core Competency Milestone Evaluation System. Despite this, Milestones tailored for anesthesiology have yet to be implemented in China. To address this gap, we have developed Chinese Anesthesiology Milestones. This study aims to assess the reliability and validity of the Chinese Anesthesiology Milestones and their correlation with objective examinations.\r\n\r\nMETHODS\r\nIn this single-center cross-sectional study, we included anesthesia residents enrolled in the standardized residency training program at our hospital during the academic year 2021 to 2022. The Chinese Anesthesiology Milestones were developed based on the American version of Anesthesiology Milestones 2.0 and the Chinese Resident Core Competency Milestone Evaluation System using the Delphi method. The Delphi panel comprised a diverse group, including education administrators, faculty from teaching hospitals, and anesthesia residents. Five attending anesthesiologists independently assessed the levels achieved by each anesthesia resident based on the Chinese Anesthesiology Milestones. Subsequently, they collaboratively discussed the ratings for each resident until a consensus was reached. The interrater reliability, internal consistency, and construct validity were assessed using Kendall's coefficient, Cronbach's α coefficient/ composite reliability, and average variance extracted, respectively. Higher values indicated better reliability or validity. The correlation between Milestone ratings and objective examination scores, including written examinations and Objective Structured Clinical Examinations, were analyzed using Pearson correlation.\r\n\r\nRESULTS\r\nThe Chinese Anesthesiology Milestones encompassed 6 competencies, including professionalism, medical knowledge and technical skills, patient care, interpersonal and communication skills, teaching ability, and life-long learning. Milestone evaluation data were available and analyzed from 66 residents. The Kendall's coefficient of concordance among raters ranged from 0.799 (95% confidence interval [CI], 0.793-0.918) to 0.942 (95% CI, 0.934-0.982). The average variance extracted, composite reliability, and Cronbach's α coefficient ranged from 0.782 to 0.920, 0.935 to 0.980, and 0.916 to 0.978, respectively. Correlations between objective examination scores and related Milestone subcompetencies were as follows: written examinations: r = 0.52 (95% CI, 0.22-0.71), technical skills stations: r = 0.51 (95% CI, 0.21-0.71), the oral test station: r = 0.66 (95% CI, 0.45-0.79), and the standardized patient station: r = 0.61 (95% CI, 0.36-0.76).\r\n\r\nCONCLUSIONS\r\nThe Chinese Anesthesiology Milestones demonstrated satisfactory interrater reliability, internal consistency, construct validity, and correlation with objective examination scores within our hospital.","PeriodicalId":7799,"journal":{"name":"Anesthesia & Analgesia","volume":"77 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Chinese Anesthesiology Milestones in Resident Evaluation: Reliability, Validity, and Correlation with Objective Examination Scores: a Cross-sectional Study.\",\"authors\":\"Xia Ruan,Xiaohan Xu,Lijian Pei,Jie Yi,Chunhua Yu,Xuerong Yu,Bo Zhu,Xiang Quan,Xu Li,Hui Jv,Yuelun Zhang,Yuguang Huang\",\"doi\":\"10.1213/ane.0000000000007279\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"BACKGROUND\\r\\nEvaluating competency acquisition during residency training is crucial. The Anesthesiology Milestones have been implemented in the United States. The China Consortium of Elite Teaching Hospitals for Residency Education has also developed the Chinese Resident Core Competency Milestone Evaluation System. Despite this, Milestones tailored for anesthesiology have yet to be implemented in China. To address this gap, we have developed Chinese Anesthesiology Milestones. This study aims to assess the reliability and validity of the Chinese Anesthesiology Milestones and their correlation with objective examinations.\\r\\n\\r\\nMETHODS\\r\\nIn this single-center cross-sectional study, we included anesthesia residents enrolled in the standardized residency training program at our hospital during the academic year 2021 to 2022. The Chinese Anesthesiology Milestones were developed based on the American version of Anesthesiology Milestones 2.0 and the Chinese Resident Core Competency Milestone Evaluation System using the Delphi method. The Delphi panel comprised a diverse group, including education administrators, faculty from teaching hospitals, and anesthesia residents. Five attending anesthesiologists independently assessed the levels achieved by each anesthesia resident based on the Chinese Anesthesiology Milestones. Subsequently, they collaboratively discussed the ratings for each resident until a consensus was reached. The interrater reliability, internal consistency, and construct validity were assessed using Kendall's coefficient, Cronbach's α coefficient/ composite reliability, and average variance extracted, respectively. Higher values indicated better reliability or validity. The correlation between Milestone ratings and objective examination scores, including written examinations and Objective Structured Clinical Examinations, were analyzed using Pearson correlation.\\r\\n\\r\\nRESULTS\\r\\nThe Chinese Anesthesiology Milestones encompassed 6 competencies, including professionalism, medical knowledge and technical skills, patient care, interpersonal and communication skills, teaching ability, and life-long learning. Milestone evaluation data were available and analyzed from 66 residents. The Kendall's coefficient of concordance among raters ranged from 0.799 (95% confidence interval [CI], 0.793-0.918) to 0.942 (95% CI, 0.934-0.982). The average variance extracted, composite reliability, and Cronbach's α coefficient ranged from 0.782 to 0.920, 0.935 to 0.980, and 0.916 to 0.978, respectively. Correlations between objective examination scores and related Milestone subcompetencies were as follows: written examinations: r = 0.52 (95% CI, 0.22-0.71), technical skills stations: r = 0.51 (95% CI, 0.21-0.71), the oral test station: r = 0.66 (95% CI, 0.45-0.79), and the standardized patient station: r = 0.61 (95% CI, 0.36-0.76).\\r\\n\\r\\nCONCLUSIONS\\r\\nThe Chinese Anesthesiology Milestones demonstrated satisfactory interrater reliability, internal consistency, construct validity, and correlation with objective examination scores within our hospital.\",\"PeriodicalId\":7799,\"journal\":{\"name\":\"Anesthesia & Analgesia\",\"volume\":\"77 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-10-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Anesthesia & Analgesia\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1213/ane.0000000000007279\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Anesthesia & Analgesia","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1213/ane.0000000000007279","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

背景评估住院医师培训期间的能力掌握情况至关重要。麻醉学里程碑已在美国实施。中国住院医师教育精英教学医院联盟也开发了中国住院医师核心能力里程碑评估系统。尽管如此,为麻醉学量身定制的里程碑尚未在中国实施。为了填补这一空白,我们开发了中国麻醉学里程碑。本研究旨在评估中国麻醉学里程碑的信度和效度及其与客观考试的相关性。方法在这项单中心横断面研究中,我们纳入了我院 2021 至 2022 学年住院医师规范化培训项目的麻醉住院医师。中国麻醉学里程碑是在美国版麻醉学里程碑 2.0 和中国住院医师核心能力里程碑评价体系的基础上,采用德尔菲法制定的。德尔菲小组由教育管理人员、教学医院教师和麻醉住院医师等不同人员组成。五位麻醉主治医师根据中国麻醉学里程碑独立评估了每位麻醉住院医师达到的水平。随后,他们共同讨论了对每位住院医师的评分,直至达成共识。使用 Kendall 系数、Cronbach's α 系数/综合信度和平均方差提取率分别评估了评分者之间的信度、内部一致性和建构效度。数值越高,表示信度或效度越高。结果中国麻醉学里程碑包括6项能力,包括职业精神、医学知识和技术技能、患者护理、人际交往和沟通技能、教学能力和终身学习。对 66 名住院医师的里程碑评估数据进行了分析。评分者之间的肯德尔一致性系数从 0.799(95% 置信区间[CI],0.793-0.918)到 0.942(95% 置信区间[CI],0.934-0.982)不等。平均方差提取率、综合信度和 Cronbach's α 系数分别为 0.782 至 0.920、0.935 至 0.980 和 0.916 至 0.978。客观考试成绩与相关 "里程碑 "子能力之间的相关性如下:笔试:r = 0.52(95% CI,0.22-0.71);技术技能站:r = 0.51(95% CI,0.21-0.71);口试站:r = 0.66(95% CI,0.45-0.79);标准化病人:r = 0.51(95% CI,0.21-0.71)。结论在我院,《中国麻醉学里程碑》显示了令人满意的考官间信度、内部一致性、建构效度以及与客观考试分数的相关性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Chinese Anesthesiology Milestones in Resident Evaluation: Reliability, Validity, and Correlation with Objective Examination Scores: a Cross-sectional Study.
BACKGROUND Evaluating competency acquisition during residency training is crucial. The Anesthesiology Milestones have been implemented in the United States. The China Consortium of Elite Teaching Hospitals for Residency Education has also developed the Chinese Resident Core Competency Milestone Evaluation System. Despite this, Milestones tailored for anesthesiology have yet to be implemented in China. To address this gap, we have developed Chinese Anesthesiology Milestones. This study aims to assess the reliability and validity of the Chinese Anesthesiology Milestones and their correlation with objective examinations. METHODS In this single-center cross-sectional study, we included anesthesia residents enrolled in the standardized residency training program at our hospital during the academic year 2021 to 2022. The Chinese Anesthesiology Milestones were developed based on the American version of Anesthesiology Milestones 2.0 and the Chinese Resident Core Competency Milestone Evaluation System using the Delphi method. The Delphi panel comprised a diverse group, including education administrators, faculty from teaching hospitals, and anesthesia residents. Five attending anesthesiologists independently assessed the levels achieved by each anesthesia resident based on the Chinese Anesthesiology Milestones. Subsequently, they collaboratively discussed the ratings for each resident until a consensus was reached. The interrater reliability, internal consistency, and construct validity were assessed using Kendall's coefficient, Cronbach's α coefficient/ composite reliability, and average variance extracted, respectively. Higher values indicated better reliability or validity. The correlation between Milestone ratings and objective examination scores, including written examinations and Objective Structured Clinical Examinations, were analyzed using Pearson correlation. RESULTS The Chinese Anesthesiology Milestones encompassed 6 competencies, including professionalism, medical knowledge and technical skills, patient care, interpersonal and communication skills, teaching ability, and life-long learning. Milestone evaluation data were available and analyzed from 66 residents. The Kendall's coefficient of concordance among raters ranged from 0.799 (95% confidence interval [CI], 0.793-0.918) to 0.942 (95% CI, 0.934-0.982). The average variance extracted, composite reliability, and Cronbach's α coefficient ranged from 0.782 to 0.920, 0.935 to 0.980, and 0.916 to 0.978, respectively. Correlations between objective examination scores and related Milestone subcompetencies were as follows: written examinations: r = 0.52 (95% CI, 0.22-0.71), technical skills stations: r = 0.51 (95% CI, 0.21-0.71), the oral test station: r = 0.66 (95% CI, 0.45-0.79), and the standardized patient station: r = 0.61 (95% CI, 0.36-0.76). CONCLUSIONS The Chinese Anesthesiology Milestones demonstrated satisfactory interrater reliability, internal consistency, construct validity, and correlation with objective examination scores within our hospital.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信