Victoria Hartwell MD, MEd, Katherine Edmunds MD, MEd, Laura Elliott MD, Brenda Williams MSN, RN, Paul T. Menk MD, Gary L. Geis MD
{"title":"在儿科急诊复苏中使用视频审查团队领导评估工具的有效性证据","authors":"Victoria Hartwell MD, MEd, Katherine Edmunds MD, MEd, Laura Elliott MD, Brenda Williams MSN, RN, Paul T. Menk MD, Gary L. Geis MD","doi":"10.1002/aet2.10985","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Effective leadership of health care action teams has demonstrated positive influence on team performance and patient care, but there is no consensus on how to assess these skills. We developed a novel team leadership assessment tool for leaders of interprofessional pediatric resuscitation teams and collected validity evidence for this tool using video review.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>This was a prospective cohort study from November 2021 to October 2022. A novel team leadership assessment tool was developed using literature review and local expertise and then piloted and refined using medical simulation. Pediatric emergency medicine (PEM) fellows from a single tertiary care pediatric medical center were enrolled, and videos of one medical resuscitation and one trauma resuscitation were collected per fellow each month. Three reviewers underwent reviewer training and then scored the videos using the assessment tool. Raters provided feedback on feasibility and ease of use using a 5-point Likert scale. Inter-rater reliability for the assessment tool using Gwet's agreement coefficient and the association between performance and clinical level of training using generalized linear mixed model were calculated.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Twelve PEM fellows enrolled and 146 videos were reviewed. The inter-rater reliability for each domain ranged from 0.45 (<i>p</i> < 0.0001) to 0.59 (<i>p</i> < 0.0001), with the inter-rater reliability of the total score being 0.49 (<i>p</i> < 0.0001). The reviewers’ mean ratings of the elements of the tool were as follows: clarity of the domains (4.6/5), the independence of each domain from each other (3.9/5), the ease of use of the 5-point Likert scale (4.5/5), the usefulness of the provided examples for each domain (4.6/5), and the ability to assess each domain without having to rewatch (4.5/5). The tool differentiated between levels of clinical training for two of the six domains (<i>p</i> < 0.02).</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>We developed a novel team leadership assessment tool for pediatric resuscitation team leaders that demonstrated moderate inter-rater reliability. The tool was easy to use and feasible for educators to assess the performance of PEM trainees in complex high-stakes clinical situations.</p>\n </section>\n </div>","PeriodicalId":37032,"journal":{"name":"AEM Education and Training","volume":"8 3","pages":""},"PeriodicalIF":1.7000,"publicationDate":"2024-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/aet2.10985","citationCount":"0","resultStr":"{\"title\":\"Validity evidence for a team-leading assessment tool in pediatric emergency resuscitations using video review\",\"authors\":\"Victoria Hartwell MD, MEd, Katherine Edmunds MD, MEd, Laura Elliott MD, Brenda Williams MSN, RN, Paul T. Menk MD, Gary L. Geis MD\",\"doi\":\"10.1002/aet2.10985\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <h3> Background</h3>\\n \\n <p>Effective leadership of health care action teams has demonstrated positive influence on team performance and patient care, but there is no consensus on how to assess these skills. We developed a novel team leadership assessment tool for leaders of interprofessional pediatric resuscitation teams and collected validity evidence for this tool using video review.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Methods</h3>\\n \\n <p>This was a prospective cohort study from November 2021 to October 2022. A novel team leadership assessment tool was developed using literature review and local expertise and then piloted and refined using medical simulation. Pediatric emergency medicine (PEM) fellows from a single tertiary care pediatric medical center were enrolled, and videos of one medical resuscitation and one trauma resuscitation were collected per fellow each month. Three reviewers underwent reviewer training and then scored the videos using the assessment tool. Raters provided feedback on feasibility and ease of use using a 5-point Likert scale. Inter-rater reliability for the assessment tool using Gwet's agreement coefficient and the association between performance and clinical level of training using generalized linear mixed model were calculated.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Results</h3>\\n \\n <p>Twelve PEM fellows enrolled and 146 videos were reviewed. The inter-rater reliability for each domain ranged from 0.45 (<i>p</i> < 0.0001) to 0.59 (<i>p</i> < 0.0001), with the inter-rater reliability of the total score being 0.49 (<i>p</i> < 0.0001). The reviewers’ mean ratings of the elements of the tool were as follows: clarity of the domains (4.6/5), the independence of each domain from each other (3.9/5), the ease of use of the 5-point Likert scale (4.5/5), the usefulness of the provided examples for each domain (4.6/5), and the ability to assess each domain without having to rewatch (4.5/5). The tool differentiated between levels of clinical training for two of the six domains (<i>p</i> < 0.02).</p>\\n </section>\\n \\n <section>\\n \\n <h3> Conclusions</h3>\\n \\n <p>We developed a novel team leadership assessment tool for pediatric resuscitation team leaders that demonstrated moderate inter-rater reliability. The tool was easy to use and feasible for educators to assess the performance of PEM trainees in complex high-stakes clinical situations.</p>\\n </section>\\n </div>\",\"PeriodicalId\":37032,\"journal\":{\"name\":\"AEM Education and Training\",\"volume\":\"8 3\",\"pages\":\"\"},\"PeriodicalIF\":1.7000,\"publicationDate\":\"2024-04-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/aet2.10985\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"AEM Education and Training\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/aet2.10985\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"EDUCATION, SCIENTIFIC DISCIPLINES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"AEM Education and Training","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/aet2.10985","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
Validity evidence for a team-leading assessment tool in pediatric emergency resuscitations using video review
Background
Effective leadership of health care action teams has demonstrated positive influence on team performance and patient care, but there is no consensus on how to assess these skills. We developed a novel team leadership assessment tool for leaders of interprofessional pediatric resuscitation teams and collected validity evidence for this tool using video review.
Methods
This was a prospective cohort study from November 2021 to October 2022. A novel team leadership assessment tool was developed using literature review and local expertise and then piloted and refined using medical simulation. Pediatric emergency medicine (PEM) fellows from a single tertiary care pediatric medical center were enrolled, and videos of one medical resuscitation and one trauma resuscitation were collected per fellow each month. Three reviewers underwent reviewer training and then scored the videos using the assessment tool. Raters provided feedback on feasibility and ease of use using a 5-point Likert scale. Inter-rater reliability for the assessment tool using Gwet's agreement coefficient and the association between performance and clinical level of training using generalized linear mixed model were calculated.
Results
Twelve PEM fellows enrolled and 146 videos were reviewed. The inter-rater reliability for each domain ranged from 0.45 (p < 0.0001) to 0.59 (p < 0.0001), with the inter-rater reliability of the total score being 0.49 (p < 0.0001). The reviewers’ mean ratings of the elements of the tool were as follows: clarity of the domains (4.6/5), the independence of each domain from each other (3.9/5), the ease of use of the 5-point Likert scale (4.5/5), the usefulness of the provided examples for each domain (4.6/5), and the ability to assess each domain without having to rewatch (4.5/5). The tool differentiated between levels of clinical training for two of the six domains (p < 0.02).
Conclusions
We developed a novel team leadership assessment tool for pediatric resuscitation team leaders that demonstrated moderate inter-rater reliability. The tool was easy to use and feasible for educators to assess the performance of PEM trainees in complex high-stakes clinical situations.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
