ASWB临床社会工作执照考试的构念无关方差:先前效度关注的重复

IF 1.7 4区社会学 Q1 SOCIAL WORK

Research on Social Work Practice Pub Date : 2023-07-12 DOI:10.1177/10497315231188305

Bryan G. Victor, Kellan McNally, Zia Qi, B. Perron

{"title":"ASWB临床社会工作执照考试的构念无关方差:先前效度关注的重复","authors":"Bryan G. Victor, Kellan McNally, Zia Qi, B. Perron","doi":"10.1177/10497315231188305","DOIUrl":null,"url":null,"abstract":"Purpose: This study sought to replicate a previous investigation of construct-irrelevant variance on the Association of Social Work Boards (ASWB) clinical licensing exam completed by Albright and Thyer over a decade ago. Method: The performance of ChatGPT was assessed on a modified version of 50 newly developed clinical exam questions currently distributed by the ASWB, where only the four multiple-choice options for each item were presented without the question. Results: ChatGPT achieved an average accuracy rate of 73.3% across three rounds of testing, providing strong evidence of construct-irrelevant variance. Discussion: These results raise concerns about the construct validity of the clinical exam and emphasize the need for reassessment of its structure and content to ensure fairness and accuracy. Based on the findings, state legislators and regulators are encouraged to temporarily discontinue the use of the ASWB exam in the clinical licensure process until its validity flaws are resolved.","PeriodicalId":47993,"journal":{"name":"Research on Social Work Practice","volume":" ","pages":""},"PeriodicalIF":1.7000,"publicationDate":"2023-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Construct-Irrelevant Variance on the ASWB Clinical Social Work Licensing Exam: A Replication of Prior Validity Concerns\",\"authors\":\"Bryan G. Victor, Kellan McNally, Zia Qi, B. Perron\",\"doi\":\"10.1177/10497315231188305\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Purpose: This study sought to replicate a previous investigation of construct-irrelevant variance on the Association of Social Work Boards (ASWB) clinical licensing exam completed by Albright and Thyer over a decade ago. Method: The performance of ChatGPT was assessed on a modified version of 50 newly developed clinical exam questions currently distributed by the ASWB, where only the four multiple-choice options for each item were presented without the question. Results: ChatGPT achieved an average accuracy rate of 73.3% across three rounds of testing, providing strong evidence of construct-irrelevant variance. Discussion: These results raise concerns about the construct validity of the clinical exam and emphasize the need for reassessment of its structure and content to ensure fairness and accuracy. Based on the findings, state legislators and regulators are encouraged to temporarily discontinue the use of the ASWB exam in the clinical licensure process until its validity flaws are resolved.\",\"PeriodicalId\":47993,\"journal\":{\"name\":\"Research on Social Work Practice\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":1.7000,\"publicationDate\":\"2023-07-12\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Research on Social Work Practice\",\"FirstCategoryId\":\"90\",\"ListUrlMain\":\"https://doi.org/10.1177/10497315231188305\",\"RegionNum\":4,\"RegionCategory\":\"社会学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"SOCIAL WORK\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Research on Social Work Practice","FirstCategoryId":"90","ListUrlMain":"https://doi.org/10.1177/10497315231188305","RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SOCIAL WORK","Score":null,"Total":0}

引用次数: 0

摘要

目的:本研究试图复制十多年前由Albright和Thyer完成的关于社会工作委员会协会(ASWB)临床许可考试中构念无关方差的调查。方法:ChatGPT的表现是在ASWB目前分发的50个新开发的临床考试问题的修改版本上进行评估的，其中每个项目只有四个选择选项，没有问题。结果:在三轮测试中，ChatGPT的平均准确率达到了73.3%，为结构无关方差提供了强有力的证据。讨论:这些结果引起了对临床考试结构效度的关注，并强调需要重新评估其结构和内容，以确保公平和准确。根据研究结果，鼓励州立法机构和监管机构在临床许可过程中暂时停止使用ASWB考试，直到其有效性缺陷得到解决。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Construct-Irrelevant Variance on the ASWB Clinical Social Work Licensing Exam: A Replication of Prior Validity Concerns

Purpose: This study sought to replicate a previous investigation of construct-irrelevant variance on the Association of Social Work Boards (ASWB) clinical licensing exam completed by Albright and Thyer over a decade ago. Method: The performance of ChatGPT was assessed on a modified version of 50 newly developed clinical exam questions currently distributed by the ASWB, where only the four multiple-choice options for each item were presented without the question. Results: ChatGPT achieved an average accuracy rate of 73.3% across three rounds of testing, providing strong evidence of construct-irrelevant variance. Discussion: These results raise concerns about the construct validity of the clinical exam and emphasize the need for reassessment of its structure and content to ensure fairness and accuracy. Based on the findings, state legislators and regulators are encouraged to temporarily discontinue the use of the ASWB exam in the clinical licensure process until its validity flaws are resolved.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Research on Social Work Practice SOCIAL WORK-

CiteScore

3.50

自引率

11.10%

发文量

105

期刊介绍： Research on Social Work Practice, sponsored by the Society for Social Work and Research, is a disciplinary journal devoted to the publication of empirical research concerning the methods and outcomes of social work practice. Social work practice is broadly interpreted to refer to the application of intentionally designed social work intervention programs to problems of societal and/or interpersonal importance, including behavior analysis or psychotherapy involving individuals; case management; practice involving couples, families, and small groups; community practice education; and the development, implementation, and evaluation of social policies.