Development and validation of a cognitive behavioral therapy for psychosis online training with automated feedback.

IF 2.6 2区 心理学 Q2 PSYCHOLOGY, CLINICAL
Psychotherapy Pub Date : 2025-03-01 DOI:10.1037/pst0000548
Sarah L Kopelovich, Rachel M Brian, Mike Tanana, Roisín Slevin, Brian Pace, Shannon K Stewart, Victoria Shepard, Dror Ben-Zeev, Scott A Baldwin, Christina S Soma, Sarah Stanco, Zac Imel
{"title":"Development and validation of a cognitive behavioral therapy for psychosis online training with automated feedback.","authors":"Sarah L Kopelovich, Rachel M Brian, Mike Tanana, Roisín Slevin, Brian Pace, Shannon K Stewart, Victoria Shepard, Dror Ben-Zeev, Scott A Baldwin, Christina S Soma, Sarah Stanco, Zac Imel","doi":"10.1037/pst0000548","DOIUrl":null,"url":null,"abstract":"<p><p>The accessibility of training and fidelity assessment is critical to implementing and sustaining empirically supported psychotherapies like cognitive behavioral therapy for psychosis (CBTp). We describe the development of an online CBTp training tool that incorporates behavioral rehearsal tasks to enable deliberate practice of cognitive and behavioral techniques for psychosis. The development process consisted of designing content, inclusive of didactics, client profiles, and learner prompts; constructing standardized performance tasks and metrics; collecting responses to learner prompts; establishing intraclass correlation (ICC) of responses among trained raters; and training a transformer-based machine learning (ML) model to meet or surpass human ICC. Authenticity ratings of each simulated client surpassed benchmarks. CBTp trainers (<i>n</i> = 12), clinicians (<i>n</i> = 78), and nonclinicians (<i>n</i> = 119) generated 3,958 unique verbal responses to 28 unique prompts (7 skills × 4 simulated clients), of which the coding team rated 1,961. Human ICC across all skills was high (mean ICC = 0.77). On average, there was a high correlation between ML and human ratings of fidelity (<i>r<sub>s</sub></i> = .74). Similarly, the average percentage of human agreement was high at 96% (range = 87%-102%), where values greater than 100 indicate that the ML model agreed with a human rater more than two human raters agreed with each other. Results suggest that it is possible to reliably measure discrete CBTp skills in response to simulated client vignettes while capturing expected variation in skill utilization across participants. These findings pave the way for a standardized, asynchronous training that incorporates automated feedback on learners' rehearsal of CBTp skills. (PsycInfo Database Record (c) 2025 APA, all rights reserved).</p>","PeriodicalId":20910,"journal":{"name":"Psychotherapy","volume":"62 1","pages":"1-11"},"PeriodicalIF":2.6000,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11921910/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychotherapy","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/pst0000548","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PSYCHOLOGY, CLINICAL","Score":null,"Total":0}
引用次数: 0

Abstract

The accessibility of training and fidelity assessment is critical to implementing and sustaining empirically supported psychotherapies like cognitive behavioral therapy for psychosis (CBTp). We describe the development of an online CBTp training tool that incorporates behavioral rehearsal tasks to enable deliberate practice of cognitive and behavioral techniques for psychosis. The development process consisted of designing content, inclusive of didactics, client profiles, and learner prompts; constructing standardized performance tasks and metrics; collecting responses to learner prompts; establishing intraclass correlation (ICC) of responses among trained raters; and training a transformer-based machine learning (ML) model to meet or surpass human ICC. Authenticity ratings of each simulated client surpassed benchmarks. CBTp trainers (n = 12), clinicians (n = 78), and nonclinicians (n = 119) generated 3,958 unique verbal responses to 28 unique prompts (7 skills × 4 simulated clients), of which the coding team rated 1,961. Human ICC across all skills was high (mean ICC = 0.77). On average, there was a high correlation between ML and human ratings of fidelity (rs = .74). Similarly, the average percentage of human agreement was high at 96% (range = 87%-102%), where values greater than 100 indicate that the ML model agreed with a human rater more than two human raters agreed with each other. Results suggest that it is possible to reliably measure discrete CBTp skills in response to simulated client vignettes while capturing expected variation in skill utilization across participants. These findings pave the way for a standardized, asynchronous training that incorporates automated feedback on learners' rehearsal of CBTp skills. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

求助全文
约1分钟内获得全文 求助全文
来源期刊
Psychotherapy
Psychotherapy PSYCHOLOGY, CLINICAL-
CiteScore
4.60
自引率
12.00%
发文量
93
期刊介绍: Psychotherapy Theory, Research, Practice, Training publishes a wide variety of articles relevant to the field of psychotherapy. The journal strives to foster interactions among individuals involved with training, practice theory, and research since all areas are essential to psychotherapy. This journal is an invaluable resource for practicing clinical and counseling psychologists, social workers, and mental health professionals.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信