Assessment of service provider competency for child and adolescent psychological treatments and psychosocial services in global mental health: evaluation of feasibility and reliability of the WeACT tool in Gaza, Palestine.
M J D Jordans, A Coetzee, H F Steen, G V Koppenol-Gonzalez, H Galayini, S Y Diab, S A Aisha, B A Kohrt
{"title":"Assessment of service provider competency for child and adolescent psychological treatments and psychosocial services in global mental health: evaluation of feasibility and reliability of the WeACT tool in Gaza, Palestine.","authors":"M J D Jordans, A Coetzee, H F Steen, G V Koppenol-Gonzalez, H Galayini, S Y Diab, S A Aisha, B A Kohrt","doi":"10.1017/gmh.2021.6","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>There is a scarcity of evaluated tools to assess whether non-specialist providers achieve minimum levels of competency to effectively and safely deliver psychological interventions in low- and middle-income countries. The objective of this study was to evaluate the reliability and utility of the newly developed Working with children - Assessment of Competencies Tool (WeACT) to assess service providers' competencies in Gaza, Palestine.</p><p><strong>Methods: </strong>The study evaluated; (1) psychometric properties of the WeACT based on observed role-plays by trainers/supervisors (<i>N</i> = 8); (2) sensitivity to change among service provider competencies (<i>N</i> = 25) using pre-and-post training WeACT scores on standardized role-plays; (3) in-service competencies among experienced service providers (<i>N</i> = 64) using standardized role-plays.</p><p><strong>Results: </strong>We demonstrated moderate interrater reliability [intraclass correlation coefficient, single measures, ICC = 0.68 (95% CI 0.48-0.86)] after practice, with high internal consistency (<i>α</i> = 0.94). WeACT assessments provided clinically relevant information on achieved levels of competencies (55% of the competencies were scored as adequate pre-training; 71% post-training; 62% in-service). Pre-post training assessment saw significant improvement in competencies (<i>W</i> = -3.64; <i>p</i> < 0.001).</p><p><strong>Conclusion: </strong>This study demonstrated positive results on the reliability and utility of the WeACT, with sufficient inter-rater agreement, excellent internal consistency, sensitivity to assess change, and providing insight needs for remedial training. The WeACT holds promise as a tool for monitoring quality of care when implementing evidence-based care at scale.</p>","PeriodicalId":520633,"journal":{"name":"Global mental health (Cambridge, England)","volume":" ","pages":"e7"},"PeriodicalIF":0.0000,"publicationDate":"2021-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1017/gmh.2021.6","citationCount":"10","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Global mental health (Cambridge, England)","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1017/gmh.2021.6","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2021/1/1 0:00:00","PubModel":"eCollection","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 10
Abstract
Background: There is a scarcity of evaluated tools to assess whether non-specialist providers achieve minimum levels of competency to effectively and safely deliver psychological interventions in low- and middle-income countries. The objective of this study was to evaluate the reliability and utility of the newly developed Working with children - Assessment of Competencies Tool (WeACT) to assess service providers' competencies in Gaza, Palestine.
Methods: The study evaluated; (1) psychometric properties of the WeACT based on observed role-plays by trainers/supervisors (N = 8); (2) sensitivity to change among service provider competencies (N = 25) using pre-and-post training WeACT scores on standardized role-plays; (3) in-service competencies among experienced service providers (N = 64) using standardized role-plays.
Results: We demonstrated moderate interrater reliability [intraclass correlation coefficient, single measures, ICC = 0.68 (95% CI 0.48-0.86)] after practice, with high internal consistency (α = 0.94). WeACT assessments provided clinically relevant information on achieved levels of competencies (55% of the competencies were scored as adequate pre-training; 71% post-training; 62% in-service). Pre-post training assessment saw significant improvement in competencies (W = -3.64; p < 0.001).
Conclusion: This study demonstrated positive results on the reliability and utility of the WeACT, with sufficient inter-rater agreement, excellent internal consistency, sensitivity to assess change, and providing insight needs for remedial training. The WeACT holds promise as a tool for monitoring quality of care when implementing evidence-based care at scale.