Automating the assessment of multicultural orientation through machine learning and natural language processing.

IF 2.6 2区 心理学 Q2 PSYCHOLOGY, CLINICAL
Psychotherapy Pub Date : 2024-02-01 DOI:10.1037/pst0000519
Simon B Goldberg, Michael Tanana, Shaakira Haywood Stewart, Camille Y Williams, Christina S Soma, David C Atkins, Zac E Imel, Jesse Owen
{"title":"Automating the assessment of multicultural orientation through machine learning and natural language processing.","authors":"Simon B Goldberg, Michael Tanana, Shaakira Haywood Stewart, Camille Y Williams, Christina S Soma, David C Atkins, Zac E Imel, Jesse Owen","doi":"10.1037/pst0000519","DOIUrl":null,"url":null,"abstract":"<p><p>Recent scholarship has highlighted the value of therapists adopting a multicultural orientation (MCO) within psychotherapy. A newly developed performance-based measure of MCO capacities exists (MCO-performance task [MCO-PT]) in which therapists respond to video-based vignettes of clients sharing culturally relevant information in therapy. The MCO-PT provides scores related to the three aspects of MCO: cultural humility (i.e., adoption of a nonsuperior and other-oriented stance toward clients), cultural opportunities (i.e., seizing or making moments in session to ask about clients' cultural identities), and cultural comfort (i.e., therapists' comfort in cultural conversations). Although a promising measure, the MCO-PT relies on labor-intensive human coding. The present study evaluated the ability to automate the scoring of the MCO-PT transcripts using modern machine learning and natural language processing methods. We included a sample of 100 participants (<i>n</i> = 613 MCO-PT responses). Results indicated that machine learning models were able to achieve near-human reliability on the average across all domains (Spearman's ρ = .75, <i>p</i> < .0001) and opportunity (ρ = .81, <i>p</i> < .0001). Performance was less robust for cultural humility (ρ = .46, <i>p</i> < .001) and was poorest for cultural comfort (ρ = .41, <i>p</i> < .001). This suggests that we may be on the cusp of being able to develop machine learning-based training paradigms that could allow therapists opportunities for feedback and deliberate practice of some key therapist behaviors, including aspects of MCO. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>","PeriodicalId":20910,"journal":{"name":"Psychotherapy","volume":" ","pages":""},"PeriodicalIF":2.6000,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11291704/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psychotherapy","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/pst0000519","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"PSYCHOLOGY, CLINICAL","Score":null,"Total":0}
引用次数: 0

Abstract

Recent scholarship has highlighted the value of therapists adopting a multicultural orientation (MCO) within psychotherapy. A newly developed performance-based measure of MCO capacities exists (MCO-performance task [MCO-PT]) in which therapists respond to video-based vignettes of clients sharing culturally relevant information in therapy. The MCO-PT provides scores related to the three aspects of MCO: cultural humility (i.e., adoption of a nonsuperior and other-oriented stance toward clients), cultural opportunities (i.e., seizing or making moments in session to ask about clients' cultural identities), and cultural comfort (i.e., therapists' comfort in cultural conversations). Although a promising measure, the MCO-PT relies on labor-intensive human coding. The present study evaluated the ability to automate the scoring of the MCO-PT transcripts using modern machine learning and natural language processing methods. We included a sample of 100 participants (n = 613 MCO-PT responses). Results indicated that machine learning models were able to achieve near-human reliability on the average across all domains (Spearman's ρ = .75, p < .0001) and opportunity (ρ = .81, p < .0001). Performance was less robust for cultural humility (ρ = .46, p < .001) and was poorest for cultural comfort (ρ = .41, p < .001). This suggests that we may be on the cusp of being able to develop machine learning-based training paradigms that could allow therapists opportunities for feedback and deliberate practice of some key therapist behaviors, including aspects of MCO. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

通过机器学习和自然语言处理自动评估多元文化取向。
最近的学术研究强调了治疗师在心理治疗中采用多元文化取向(MCO)的价值。新近开发了一种基于表现的 MCO 能力测量方法(MCO-表现任务 [MCO-PT]),治疗师可在其中对客户在治疗中分享文化相关信息的视频片段做出反应。MCO-PT 提供了与 MCO 的三个方面相关的分数:文化谦逊(即对客户采取不卑不亢和以他人为导向的立场)、文化机会(即在治疗过程中抓住或创造时机询问客户的文化身份)和文化舒适(即治疗师在文化对话中的舒适度)。尽管 MCO-PT 是一项很有前途的测量方法,但它依赖于劳动密集型的人工编码。本研究评估了使用现代机器学习和自然语言处理方法对 MCO-PT 记录进行自动评分的能力。我们纳入了 100 名参与者的样本(n = 613 个 MCO-PT 回答)。结果表明,机器学习模型能够在所有领域(Spearman's ρ = .75,p < .0001)和机会(ρ = .81,p < .0001)平均达到接近人类的可靠性。文化谦逊方面的表现不那么稳健(ρ = .46,p < .001),文化舒适方面的表现最差(ρ = .41,p < .001)。这表明,我们可能即将开发出基于机器学习的培训范例,让治疗师有机会获得反馈,并有意识地练习一些关键的治疗师行为,包括 MCO 的各个方面。(PsycInfo Database Record (c) 2024 APA,保留所有权利)。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Psychotherapy
Psychotherapy PSYCHOLOGY, CLINICAL-
CiteScore
4.60
自引率
12.00%
发文量
93
期刊介绍: Psychotherapy Theory, Research, Practice, Training publishes a wide variety of articles relevant to the field of psychotherapy. The journal strives to foster interactions among individuals involved with training, practice theory, and research since all areas are essential to psychotherapy. This journal is an invaluable resource for practicing clinical and counseling psychologists, social workers, and mental health professionals.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信