Initial evaluation of an AI-augmented progress monitoring and outcome assessment

IF 1.2 Q3 PSYCHOLOGY, CLINICAL

Counselling & Psychotherapy Research Pub Date : 2024-12-26 DOI:10.1002/capr.12869

Scott T. Meier

{"title":"Initial evaluation of an AI-augmented progress monitoring and outcome assessment","authors":"Scott T. Meier","doi":"10.1002/capr.12869","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Purpose</h3>\n \n <p>Most mental health providers have yet to adopt progress monitoring and outcome assessment (PMOA) measures. Although a variety of explanations have been proposed in the literature, a key reason is the burden of time and effort necessary for clients and clinicians to complete, interpret and apply the results of PMOA measures. This evaluation explores the feasibility and initial results of employing ChatGPT to analyse clinicians' unstructured session progress notes for PMOA.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>Using a simulated patient with 17 trainee therapists, the study examined whether artificial intelligence (AI) can assist in generating thematic summaries relevant to clinical progress and outcomes. Therapists' session summaries were combined to evaluate the continuation of key clinical themes across four sessions for a simulated patient. Trainees also provided brief quantitative ratings per session about the patient's working alliance, negative affect (NA), avoidance of NA and levels of distress.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>AI-generated results found a (a) persistent focus across sessions regarding the patient's relationship issues with an abusive caretaker, reluctance to disclose and avoidance of NA, and (b) substantial convergence between human-generated and AI-generated thematic summaries.</p>\n </section>\n \n <section>\n \n <h3> Discussion</h3>\n \n <p>Overall, the use of AI to analyse clinical progress notes appears feasible and psychometrically sound. By minimising resources needed by patients and clinicians to produce clinically relevant data, an AI-augmented approach can reduce a major obstacle to clinicians' adoption of PMOA measures for feedback purposes.</p>\n </section>\n </div>","PeriodicalId":46997,"journal":{"name":"Counselling & Psychotherapy Research","volume":"25 1","pages":""},"PeriodicalIF":1.2000,"publicationDate":"2024-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Counselling & Psychotherapy Research","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/capr.12869","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"PSYCHOLOGY, CLINICAL","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose

Most mental health providers have yet to adopt progress monitoring and outcome assessment (PMOA) measures. Although a variety of explanations have been proposed in the literature, a key reason is the burden of time and effort necessary for clients and clinicians to complete, interpret and apply the results of PMOA measures. This evaluation explores the feasibility and initial results of employing ChatGPT to analyse clinicians' unstructured session progress notes for PMOA.

Methods

Using a simulated patient with 17 trainee therapists, the study examined whether artificial intelligence (AI) can assist in generating thematic summaries relevant to clinical progress and outcomes. Therapists' session summaries were combined to evaluate the continuation of key clinical themes across four sessions for a simulated patient. Trainees also provided brief quantitative ratings per session about the patient's working alliance, negative affect (NA), avoidance of NA and levels of distress.

Results

AI-generated results found a (a) persistent focus across sessions regarding the patient's relationship issues with an abusive caretaker, reluctance to disclose and avoidance of NA, and (b) substantial convergence between human-generated and AI-generated thematic summaries.

Discussion

Overall, the use of AI to analyse clinical progress notes appears feasible and psychometrically sound. By minimising resources needed by patients and clinicians to produce clinically relevant data, an AI-augmented approach can reduce a major obstacle to clinicians' adoption of PMOA measures for feedback purposes.

查看原文本刊更多论文

对人工智能增强的进展监测和结果评估进行初步评估

大多数精神卫生服务提供者尚未采用进展监测和结果评估（PMOA）措施。虽然文献中提出了各种各样的解释，但一个关键原因是客户和临床医生完成、解释和应用PMOA测量结果所需的时间和精力负担。本评估探讨了使用ChatGPT分析临床医生PMOA非结构化会话进度记录的可行性和初步结果。方法通过17名实习治疗师的模拟患者，研究人工智能（AI）是否可以帮助生成与临床进展和结果相关的主题摘要。治疗师的会议总结结合评估关键临床主题的延续跨越四个会议模拟病人。学员还提供了关于病人的工作联盟、负面影响（NA）、避免负面影响和痛苦程度的简短定量评分。人工智能生成的结果发现(a)患者与虐待性看护人的关系问题持续关注，不愿透露和避免NA，以及(b)人类生成和人工智能生成的主题摘要之间存在实质性趋同。总的来说，使用人工智能来分析临床进展记录似乎是可行的，并且在心理测量学上是合理的。通过最大限度地减少患者和临床医生产生临床相关数据所需的资源，人工智能增强方法可以减少临床医生采用PMOA措施以获得反馈的主要障碍。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Counselling & Psychotherapy Research PSYCHOLOGY, CLINICAL-

CiteScore

4.40

自引率

12.50%

发文量

期刊介绍： Counselling and Psychotherapy Research is an innovative international peer-reviewed journal dedicated to linking research with practice. Pluralist in orientation, the journal recognises the value of qualitative, quantitative and mixed methods strategies of inquiry and aims to promote high-quality, ethical research that informs and develops counselling and psychotherapy practice. CPR is a journal of the British Association of Counselling and Psychotherapy, promoting reflexive research strongly linked to practice. The journal has its own website: www.cprjournal.com. The aim of this site is to further develop links between counselling and psychotherapy research and practice by offering accessible information about both the specific contents of each issue of CPR, as well as wider developments in counselling and psychotherapy research. The aims are to ensure that research remains relevant to practice, and for practice to continue to inform research development.