评价特征观察工具的可靠性评估

IF 1.4 3区 心理学 Q3 EDUCATION, SPECIAL
Andrew M. Markelz, Benjamin S. Riden, Kimberly A. Zoder-Martell, Joseph E. Miller, Sarah J. Bolinger
{"title":"评价特征观察工具的可靠性评估","authors":"Andrew M. Markelz, Benjamin S. Riden, Kimberly A. Zoder-Martell, Joseph E. Miller, Sarah J. Bolinger","doi":"10.1177/1098300720907988","DOIUrl":null,"url":null,"abstract":"Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos (n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of .80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p < .001, and mean kappa score of .91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.","PeriodicalId":47652,"journal":{"name":"Journal of Positive Behavior Interventions","volume":null,"pages":null},"PeriodicalIF":1.4000,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1098300720907988","citationCount":"6","resultStr":"{\"title\":\"Reliability Assessment of an Observation Tool to Measure Praise Characteristics\",\"authors\":\"Andrew M. Markelz, Benjamin S. Riden, Kimberly A. Zoder-Martell, Joseph E. Miller, Sarah J. Bolinger\",\"doi\":\"10.1177/1098300720907988\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos (n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of .80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p < .001, and mean kappa score of .91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.\",\"PeriodicalId\":47652,\"journal\":{\"name\":\"Journal of Positive Behavior Interventions\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.4000,\"publicationDate\":\"2021-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://sci-hub-pdf.com/10.1177/1098300720907988\",\"citationCount\":\"6\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Positive Behavior Interventions\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://doi.org/10.1177/1098300720907988\",\"RegionNum\":3,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"EDUCATION, SPECIAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Positive Behavior Interventions","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/1098300720907988","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"EDUCATION, SPECIAL","Score":null,"Total":0}
引用次数: 6

摘要

在数十年来对表扬及其对学生行为影响的研究的支持下,我们开发了行为特异性表扬-观察工具(BSP-OT)来衡量有效表扬的特征。我们使用组内相关性(ICC)和Cohen’s kappa统计来评估BSP-OT的评估者间可靠性,以衡量表扬的特异性、偶然性和多样性。此外,我们通过社会有效性测量评估了该工具的有用性和实用性。四位有表扬研究经验的评分者完成了一项调查,并对真实情况下的教师进行了视频编码(n=14)。总体评估表明,ICC(2,k)为.80:95%置信区间(CI)=[0.77,0.83],F(26919906)=5.1,p<.001,平均kappa评分为.91的评分者之间具有很强的可靠性。此外,高社会有效性评级表明,BSP-OT对表扬研究和教师发展领域做出了宝贵贡献。介绍了BSP-OT的开发过程和研究结果,并讨论了对未来研究的启示和建议。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Reliability Assessment of an Observation Tool to Measure Praise Characteristics
Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos (n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of .80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p < .001, and mean kappa score of .91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
4.90
自引率
14.30%
发文量
10
期刊介绍: ...offers sound, research-based principles of positive behavior support for use in school, home and community settings with people with challenges in behavioral adaptation. Regular features include empirical research; discussion, literature reviews, and conceptual papers; programs, practices, and innovations; forum; and media reviews.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信