Examining the reliability of the administrative case review process of a state child welfare system

IF 3.4 2区心理学 Q1 FAMILY STUDIES

Child Abuse & Neglect Pub Date : 2025-02-14 DOI:10.1016/j.chiabu.2025.107305

Cody Oltmans, Ka Ho Brian Chor

{"title":"Examining the reliability of the administrative case review process of a state child welfare system","authors":"Cody Oltmans, Ka Ho Brian Chor","doi":"10.1016/j.chiabu.2025.107305","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>The Adoption Assistance and Child Welfare Act mandates an Administrative Case Review (ACR) for every child in foster care every 6 months regarding their safety, adherence to case planning, and progress towards permanency. Despite limited research, a well-implemented ACR process should improve child welfare outcomes.</div></div><div><h3>Objective</h3><div>To systematically examine the reliability of one state's ACR process.</div></div><div><h3>Participants and Setting</h3><div>In a large Midwestern state child welfare system, the authors partnered with the state child welfare agency's ACR administrators and five independent, external experts to examine the reliability of their ACR ratings across key priority areas.</div></div><div><h3>Methods</h3><div>Intraclass correlation coefficients (ICCs) and percentage agreement on ACR ratings between the ACR reviewers and external experts were calculated. In December 2021 – May 2022 (Round 1) and December 2022 – May 2023 (Round 2), a random representative sample of 290 foster care cases were selected whose ACRs were completed by the ACR reviewers and external experts. Results from Round 1 informed documentation and training efforts prior to Round 2.</div></div><div><h3>Results</h3><div>The ICCs and percentage agreement on four combined priority areas in Round 1 was 0.43 and 40 %, respectively, compared to 0.63 and 63 %, respectively, in Round 2 on five combined priority areas. Both rounds showed variability in reliability across priority areas.</div></div><div><h3>Conclusions</h3><div>Reliability improved from Round 1 to Round 2. Rigorous examination of the reliability of the ACR process, in addition to compliance with federal legislation, should be the basis for states to harness the full capacity of ACR to improve child welfare outcomes.</div></div>","PeriodicalId":51343,"journal":{"name":"Child Abuse & Neglect","volume":"161 ","pages":"Article 107305"},"PeriodicalIF":3.4000,"publicationDate":"2025-02-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Child Abuse & Neglect","FirstCategoryId":"102","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0145213425000602","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"FAMILY STUDIES","Score":null,"Total":0}

引用次数: 0

Abstract

Background

The Adoption Assistance and Child Welfare Act mandates an Administrative Case Review (ACR) for every child in foster care every 6 months regarding their safety, adherence to case planning, and progress towards permanency. Despite limited research, a well-implemented ACR process should improve child welfare outcomes.

Objective

To systematically examine the reliability of one state's ACR process.

Participants and Setting

In a large Midwestern state child welfare system, the authors partnered with the state child welfare agency's ACR administrators and five independent, external experts to examine the reliability of their ACR ratings across key priority areas.

Methods

Intraclass correlation coefficients (ICCs) and percentage agreement on ACR ratings between the ACR reviewers and external experts were calculated. In December 2021 – May 2022 (Round 1) and December 2022 – May 2023 (Round 2), a random representative sample of 290 foster care cases were selected whose ACRs were completed by the ACR reviewers and external experts. Results from Round 1 informed documentation and training efforts prior to Round 2.

Results

The ICCs and percentage agreement on four combined priority areas in Round 1 was 0.43 and 40 %, respectively, compared to 0.63 and 63 %, respectively, in Round 2 on five combined priority areas. Both rounds showed variability in reliability across priority areas.

Conclusions

Reliability improved from Round 1 to Round 2. Rigorous examination of the reliability of the ACR process, in addition to compliance with federal legislation, should be the basis for states to harness the full capacity of ACR to improve child welfare outcomes.

查看原文本刊更多论文

考察国家儿童福利制度行政案件审查程序的可靠性

《收养援助和儿童福利法》要求每6个月对每个寄养儿童进行一次行政案件审查（ACR），审查内容包括他们的安全、对案件计划的遵守情况以及迈向永久收养的进展情况。尽管研究有限，但良好实施的ACR程序应能改善儿童福利结果。目的系统地考察一州ACR流程的可靠性。参与者和背景在中西部一个大型的州儿童福利系统中，作者与州儿童福利机构的ACR管理人员和五位独立的外部专家合作，检查他们在关键优先领域的ACR评级的可靠性。方法计算ACR审稿人与外部专家对ACR评分的相关系数（ICCs）和一致性百分比。在2021年12月至2022年5月（第1轮）和2022年12月至2023年5月（第2轮），随机抽取290个寄养病例的代表性样本，这些病例的ACR由ACR审稿人和外部专家完成。第一轮的结果为第二轮之前的文件和培训工作提供了信息。结果第1轮4个联合优先领域的ICCs和百分比一致性分别为0.43%和40%，而第2轮5个联合优先领域的ICCs和百分比一致性分别为0.63%和63%。两轮都显示了不同优先领域的可靠性差异。结论从第1轮到第2轮，可靠性有所提高。除了遵守联邦立法外，对儿童福利制度程序的可靠性进行严格审查应该是各州充分利用儿童福利制度的能力来改善儿童福利结果的基础。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Child Abuse & Neglect Multiple-

CiteScore

7.40

自引率

10.40%

发文量

397

期刊介绍： Official Publication of the International Society for Prevention of Child Abuse and Neglect. Child Abuse & Neglect The International Journal, provides an international, multidisciplinary forum on all aspects of child abuse and neglect, with special emphasis on prevention and treatment; the scope extends further to all those aspects of life which either favor or hinder child development. While contributions will primarily be from the fields of psychology, psychiatry, social work, medicine, nursing, law enforcement, legislature, education, and anthropology, the Journal encourages the concerned lay individual and child-oriented advocate organizations to contribute.