Prognostic predictions in psychosis: exploring the complementary role of machine learning models.

0 PSYCHIATRY
Violet van Dee, Seyed M Kia, Caterina Fregosi, Wilma E Swildens, Anne Alkema, Albert Batalla, Coen van den Berg, Danko Coric, Edwin van Dellen, Lotte G Dijkstra, Arthur van den Doel, Livia S Dominicus, John Enterman, Frank L Gerritse, Marte Z van der Horst, Fedor van Houwelingen, Charlotte S Koch, Lisanne E M Koomen, Marjan Kromkamp, Michelle Lancee, Brian E Mouthaan, Diane F van Rappard, Eline J Regeer, Raymond W J Salet, Metten Somers, Jorgen Straalman, Marjolein H T de Vette, Judith Voogt, Inge Winter-van Rossum, Rene S Kahn, Wiepke Cahn, Hugo G Schnack
{"title":"Prognostic predictions in psychosis: exploring the complementary role of machine learning models.","authors":"Violet van Dee, Seyed M Kia, Caterina Fregosi, Wilma E Swildens, Anne Alkema, Albert Batalla, Coen van den Berg, Danko Coric, Edwin van Dellen, Lotte G Dijkstra, Arthur van den Doel, Livia S Dominicus, John Enterman, Frank L Gerritse, Marte Z van der Horst, Fedor van Houwelingen, Charlotte S Koch, Lisanne E M Koomen, Marjan Kromkamp, Michelle Lancee, Brian E Mouthaan, Diane F van Rappard, Eline J Regeer, Raymond W J Salet, Metten Somers, Jorgen Straalman, Marjolein H T de Vette, Judith Voogt, Inge Winter-van Rossum, Rene S Kahn, Wiepke Cahn, Hugo G Schnack","doi":"10.1136/bmjment-2025-301594","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Predicting outcomes in schizophrenia spectrum disorders is challenging due to the variability of individual trajectories. While machine learning (ML) shows promise in outcome prediction, it has not yet been integrated into clinical practice. Understanding how ML models (MLMs) can complement psychiatrists' predictions and bridge the gap between MLM capabilities and practical use is key.</p><p><strong>Objective: </strong>This vignette study aims to compare the performance of psychiatrists and MLMs in predicting short-term symptomatic and functional remission in patients with first-episode psychosis and explore whether MLMs can improve psychiatrists' prognostic accuracy.</p><p><strong>Method: </strong>24 psychiatrists predicted symptomatic and functional remission probabilities at 10 weeks based on written baseline information from 66 patients in the OPtimization of Treatment and Management of Schizophrenia in Europe (OPTiMiSE) trial. ML-generated predictions based on these vignettes were then shared with psychiatrists, allowing them to adjust their estimates.</p><p><strong>Findings: </strong>The predictive accuracy of the MLM was low but comparable to that of psychiatrists for symptomatic remission (MLM: 0.50, psychiatrists: 0.52) and comparable to that of psychiatrists for functional remission (MLM: 0.72, psychiatrists: 0.79). Inter-rater agreement was low but comparable for psychiatrists and the MLM. Although the MLM did not improve overall predictive accuracy, it showed potential in aiding psychiatrists with difficult-to-predict cases. However, psychiatrists struggled to recognise when to rely on the model's output, and we were unable to determine a clear pattern in these cases based on their characteristics.</p><p><strong>Conclusions: </strong>MLMs may have the potential to support psychiatric decision-making, particularly in difficult-to-predict cases, but at present, their effectiveness remains limited due to constraints in predictive accuracy and the ability to identify when to rely on the model's output. Addressing these issues is crucial to improve the utility of MLMs and foster their integration into clinical practice.</p><p><strong>Clinical implications: </strong>MLMs are best suited as supplementary tools, providing a second opinion while psychiatrists retain decision-making autonomy. Integrating predictions from both sources may help reduce individual biases and improve accuracy. This approach leverages the strengths of MLMs without compromising clinical responsibility.</p>","PeriodicalId":72434,"journal":{"name":"BMJ mental health","volume":"28 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2025-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMJ mental health","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1136/bmjment-2025-301594","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"PSYCHIATRY","Score":null,"Total":0}
引用次数: 0

Abstract

Background: Predicting outcomes in schizophrenia spectrum disorders is challenging due to the variability of individual trajectories. While machine learning (ML) shows promise in outcome prediction, it has not yet been integrated into clinical practice. Understanding how ML models (MLMs) can complement psychiatrists' predictions and bridge the gap between MLM capabilities and practical use is key.

Objective: This vignette study aims to compare the performance of psychiatrists and MLMs in predicting short-term symptomatic and functional remission in patients with first-episode psychosis and explore whether MLMs can improve psychiatrists' prognostic accuracy.

Method: 24 psychiatrists predicted symptomatic and functional remission probabilities at 10 weeks based on written baseline information from 66 patients in the OPtimization of Treatment and Management of Schizophrenia in Europe (OPTiMiSE) trial. ML-generated predictions based on these vignettes were then shared with psychiatrists, allowing them to adjust their estimates.

Findings: The predictive accuracy of the MLM was low but comparable to that of psychiatrists for symptomatic remission (MLM: 0.50, psychiatrists: 0.52) and comparable to that of psychiatrists for functional remission (MLM: 0.72, psychiatrists: 0.79). Inter-rater agreement was low but comparable for psychiatrists and the MLM. Although the MLM did not improve overall predictive accuracy, it showed potential in aiding psychiatrists with difficult-to-predict cases. However, psychiatrists struggled to recognise when to rely on the model's output, and we were unable to determine a clear pattern in these cases based on their characteristics.

Conclusions: MLMs may have the potential to support psychiatric decision-making, particularly in difficult-to-predict cases, but at present, their effectiveness remains limited due to constraints in predictive accuracy and the ability to identify when to rely on the model's output. Addressing these issues is crucial to improve the utility of MLMs and foster their integration into clinical practice.

Clinical implications: MLMs are best suited as supplementary tools, providing a second opinion while psychiatrists retain decision-making autonomy. Integrating predictions from both sources may help reduce individual biases and improve accuracy. This approach leverages the strengths of MLMs without compromising clinical responsibility.

精神病的预后预测:探索机器学习模型的补充作用。
背景:由于个体轨迹的可变性,预测精神分裂症谱系障碍的预后具有挑战性。虽然机器学习(ML)在结果预测方面显示出前景,但它尚未融入临床实践。理解机器学习模型(传销)如何补充精神科医生的预测,并弥合传销能力与实际应用之间的差距是关键。目的:本研究旨在比较精神科医生和传销媒介在预测首发精神病患者短期症状和功能缓解方面的表现,并探讨传销媒介是否能提高精神科医生预测的准确性。方法:24名精神科医生根据来自66名欧洲精神分裂症治疗和管理优化(OPTiMiSE)试验患者的书面基线信息,预测10周时症状和功能缓解的可能性。然后,基于这些小插曲的ml生成的预测与精神科医生分享,允许他们调整他们的估计。结果:MLM的预测准确度较低,但与精神科医生对症状缓解的预测准确度相当(MLM: 0.50,精神科医生:0.52),与精神科医生对功能缓解的预测准确度相当(MLM: 0.72,精神科医生:0.79)。评价间的一致性较低,但在精神科医生和传销之间具有可比性。尽管传销并没有提高总体预测的准确性,但它在帮助精神病学家处理难以预测的病例方面显示出了潜力。然而,精神科医生很难识别何时依赖模型的输出,我们无法根据这些病例的特征确定一个明确的模式。结论:传销可能具有支持精神病学决策的潜力,特别是在难以预测的病例中,但目前,由于预测准确性和识别何时依赖模型输出的能力的限制,其有效性仍然有限。解决这些问题对于提高传销的效用和促进其融入临床实践至关重要。临床意义:传销是最适合作为补充工具,提供第二意见,而精神科医生保留决策自主权。整合两种来源的预测可能有助于减少个人偏见并提高准确性。这种方法在不损害临床责任的情况下利用了传销的优势。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
6.80
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信