The effect of measurement error on the positive predictive value of PSW methods for SLD identification: How buffer zones dispel the illusion of inaccuracy

IF 3.8 1区 心理学 Q1 PSYCHOLOGY, SOCIAL
W. Joel Schneider , Dawn P. Flanagan , Christopher R. Niileksela , Joseph R. Engler
{"title":"The effect of measurement error on the positive predictive value of PSW methods for SLD identification: How buffer zones dispel the illusion of inaccuracy","authors":"W. Joel Schneider ,&nbsp;Dawn P. Flanagan ,&nbsp;Christopher R. Niileksela ,&nbsp;Joseph R. Engler","doi":"10.1016/j.jsp.2023.101280","DOIUrl":null,"url":null,"abstract":"<div><p>Patterns of Strengths and Weaknesses (PSW) methods are widely used for identifying specific learning disabilities (SLDs). Several researchers, however, have reported that the diagnostic accuracy of PSW methods is unacceptably low when strict thresholds were used to identify students with SLDs. We believe these findings give a misleading impression of the magnitude of the diagnostic errors that are likely to arise in PSW assessments. In a simulation study of 10 million cases using a simplified PSW method for demonstration, most of what have been called diagnostic errors were cases in which observed scores and true scores fell on opposite sides of a strict threshold but were still within a buffer zone the size of a typical measurement error. Because small score differences do not result in meaningfully different case conceptualizations, the use of buffer zones reveals that previous estimates of the diagnostic accuracy of PSW methods are misleadingly low. We also demonstrate that diagnostic decisions become increasingly reliable when observed scores are comfortably distant from diagnostic thresholds. For practitioners, we present a flowchart and practical guidelines to improve the accuracy and stability of SLD identification decisions.</p></div>","PeriodicalId":48232,"journal":{"name":"Journal of School Psychology","volume":null,"pages":null},"PeriodicalIF":3.8000,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of School Psychology","FirstCategoryId":"102","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0022440523001085","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, SOCIAL","Score":null,"Total":0}
引用次数: 0

Abstract

Patterns of Strengths and Weaknesses (PSW) methods are widely used for identifying specific learning disabilities (SLDs). Several researchers, however, have reported that the diagnostic accuracy of PSW methods is unacceptably low when strict thresholds were used to identify students with SLDs. We believe these findings give a misleading impression of the magnitude of the diagnostic errors that are likely to arise in PSW assessments. In a simulation study of 10 million cases using a simplified PSW method for demonstration, most of what have been called diagnostic errors were cases in which observed scores and true scores fell on opposite sides of a strict threshold but were still within a buffer zone the size of a typical measurement error. Because small score differences do not result in meaningfully different case conceptualizations, the use of buffer zones reveals that previous estimates of the diagnostic accuracy of PSW methods are misleadingly low. We also demonstrate that diagnostic decisions become increasingly reliable when observed scores are comfortably distant from diagnostic thresholds. For practitioners, we present a flowchart and practical guidelines to improve the accuracy and stability of SLD identification decisions.

测量误差对用于 SLD 识别的 PSW 方法的阳性预测值的影响:缓冲区如何消除不准确的错觉
优势和劣势模式(PSW)方法被广泛用于识别特殊学习障碍(SLD)。然而,一些研究人员报告说,当使用严格的阈值来识别有特殊学习障碍的学生时,PSW 方法的诊断准确性低得令人无法接受。我们认为,这些研究结果误导了人们对 PSW 评估中可能出现的诊断误差的认识。在一项使用简化的 PSW 方法对 1000 万个案例进行演示的模拟研究中,大多数所谓的诊断误差是指观察分数和真实分数落在严格阈值的两侧,但仍在典型测量误差大小的缓冲区内。由于微小的分数差异不会导致有意义的病例概念差异,缓冲区的使用揭示了以前对 PSW 方法诊断准确性的估计过低,具有误导性。我们还证明,当观察到的分数与诊断阈值相距较远时,诊断决定会变得越来越可靠。我们为从业人员提供了一个流程图和实用指南,以提高 SLD 识别决策的准确性和稳定性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of School Psychology
Journal of School Psychology PSYCHOLOGY, EDUCATIONAL-
CiteScore
6.70
自引率
8.00%
发文量
71
期刊介绍: The Journal of School Psychology publishes original empirical articles and critical reviews of the literature on research and practices relevant to psychological and behavioral processes in school settings. JSP presents research on intervention mechanisms and approaches; schooling effects on the development of social, cognitive, mental-health, and achievement-related outcomes; assessment; and consultation. Submissions from a variety of disciplines are encouraged. All manuscripts are read by the Editor and one or more editorial consultants with the intent of providing appropriate and constructive written reviews.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信