Franklin Dexter, Bradley J. Hindman, Kokila N. Thenuwara
{"title":"运用科克伦Q测验评估护士麻醉师工作习惯量表中单项得分的信息含量,用于提供反馈","authors":"Franklin Dexter, Bradley J. Hindman, Kokila N. Thenuwara","doi":"10.1016/j.pcorm.2025.100519","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Annual ongoing professional practice evaluation (i.e., peer review) is mandatory for nurse anesthetists at many hospitals. A work habits instrument is a valid and psychometrically reliable tool for these high-stakes assessments. Our study focuses on item-specific feedback for nurse anesthetists to increase their scores.</div></div><div><h3>Methods</h3><div>The retrospective cohort study used all 9 complete academic years for which the studied department used the scale, July 2015 through June 2024. There were 59,863 evaluations of 713 nurse anesthetist years provided by 167 rating anesthesiologists, each evaluation with 6 non-Likert scored items from one to five.</div></div><div><h3>Results</h3><div>The work habits instrument functioned as a binary (5 vs ≤4) multivariate (6-item) correlated (Cronbach alpha 0.96) array. Consequently, the 90 % (53,626/59,863) of evaluations with all 6 items five, or all 6 items scored less than five, provided information about the nurse anesthetists' performance quality but no potentially useful item-specific information for feedback. Cochran Q tests were performed for each of the nurse anesthetist years using the remaining 10 % of evaluations to look for items differing significantly from the other items. There were 4.6 % (33/713) of nurse anesthetist years with adjusted P < 0.05, representing 1.5 % (911/59,863) of evaluations. The 4.6 % did not differ from the nominal Type I error rate of 5 % (P = 0.73).</div></div><div><h3>Conclusions</h3><div>Providing analyses of individual item scores could benefit, at most, 5 % of nurse anesthetists, likely fewer. Nurse anesthetists with low ongoing professional practice evaluations should have focused evaluation wholistically based on work habits.</div></div>","PeriodicalId":53468,"journal":{"name":"Perioperative Care and Operating Room Management","volume":"40 ","pages":"Article 100519"},"PeriodicalIF":0.0000,"publicationDate":"2025-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Evaluation using Cochran’s Q tests of the information content of individual items’ scores in the nurse anesthetist work habits instrument for use in providing feedback\",\"authors\":\"Franklin Dexter, Bradley J. Hindman, Kokila N. Thenuwara\",\"doi\":\"10.1016/j.pcorm.2025.100519\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background</h3><div>Annual ongoing professional practice evaluation (i.e., peer review) is mandatory for nurse anesthetists at many hospitals. A work habits instrument is a valid and psychometrically reliable tool for these high-stakes assessments. Our study focuses on item-specific feedback for nurse anesthetists to increase their scores.</div></div><div><h3>Methods</h3><div>The retrospective cohort study used all 9 complete academic years for which the studied department used the scale, July 2015 through June 2024. There were 59,863 evaluations of 713 nurse anesthetist years provided by 167 rating anesthesiologists, each evaluation with 6 non-Likert scored items from one to five.</div></div><div><h3>Results</h3><div>The work habits instrument functioned as a binary (5 vs ≤4) multivariate (6-item) correlated (Cronbach alpha 0.96) array. Consequently, the 90 % (53,626/59,863) of evaluations with all 6 items five, or all 6 items scored less than five, provided information about the nurse anesthetists' performance quality but no potentially useful item-specific information for feedback. Cochran Q tests were performed for each of the nurse anesthetist years using the remaining 10 % of evaluations to look for items differing significantly from the other items. There were 4.6 % (33/713) of nurse anesthetist years with adjusted P < 0.05, representing 1.5 % (911/59,863) of evaluations. The 4.6 % did not differ from the nominal Type I error rate of 5 % (P = 0.73).</div></div><div><h3>Conclusions</h3><div>Providing analyses of individual item scores could benefit, at most, 5 % of nurse anesthetists, likely fewer. Nurse anesthetists with low ongoing professional practice evaluations should have focused evaluation wholistically based on work habits.</div></div>\",\"PeriodicalId\":53468,\"journal\":{\"name\":\"Perioperative Care and Operating Room Management\",\"volume\":\"40 \",\"pages\":\"Article 100519\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-06-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Perioperative Care and Operating Room Management\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2405603025000603\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Nursing\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Perioperative Care and Operating Room Management","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2405603025000603","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Nursing","Score":null,"Total":0}
Evaluation using Cochran’s Q tests of the information content of individual items’ scores in the nurse anesthetist work habits instrument for use in providing feedback
Background
Annual ongoing professional practice evaluation (i.e., peer review) is mandatory for nurse anesthetists at many hospitals. A work habits instrument is a valid and psychometrically reliable tool for these high-stakes assessments. Our study focuses on item-specific feedback for nurse anesthetists to increase their scores.
Methods
The retrospective cohort study used all 9 complete academic years for which the studied department used the scale, July 2015 through June 2024. There were 59,863 evaluations of 713 nurse anesthetist years provided by 167 rating anesthesiologists, each evaluation with 6 non-Likert scored items from one to five.
Results
The work habits instrument functioned as a binary (5 vs ≤4) multivariate (6-item) correlated (Cronbach alpha 0.96) array. Consequently, the 90 % (53,626/59,863) of evaluations with all 6 items five, or all 6 items scored less than five, provided information about the nurse anesthetists' performance quality but no potentially useful item-specific information for feedback. Cochran Q tests were performed for each of the nurse anesthetist years using the remaining 10 % of evaluations to look for items differing significantly from the other items. There were 4.6 % (33/713) of nurse anesthetist years with adjusted P < 0.05, representing 1.5 % (911/59,863) of evaluations. The 4.6 % did not differ from the nominal Type I error rate of 5 % (P = 0.73).
Conclusions
Providing analyses of individual item scores could benefit, at most, 5 % of nurse anesthetists, likely fewer. Nurse anesthetists with low ongoing professional practice evaluations should have focused evaluation wholistically based on work habits.
期刊介绍:
The objective of this new online journal is to serve as a multidisciplinary, peer-reviewed source of information related to the administrative, economic, operational, safety, and quality aspects of the ambulatory and in-patient operating room and interventional procedural processes. The journal will provide high-quality information and research findings on operational and system-based approaches to ensure safe, coordinated, and high-value periprocedural care. With the current focus on value in health care it is essential that there is a venue for researchers to publish articles on quality improvement process initiatives, process flow modeling, information management, efficient design, cost improvement, use of novel technologies, and management.