Receiving Information on Machine Learning-Based Clinical Decision Support Systems in Psychiatric Services Increases Staff Trust in These Systems: A Randomized Survey Experiment.

IF 5.3 2区 医学 Q1 PSYCHIATRY
Erik Perfalk, Martin Bernstorff, Andreas Aalkjær Danielsen, Søren Dinesen Østergaard
{"title":"Receiving Information on Machine Learning-Based Clinical Decision Support Systems in Psychiatric Services Increases Staff Trust in These Systems: A Randomized Survey Experiment.","authors":"Erik Perfalk, Martin Bernstorff, Andreas Aalkjær Danielsen, Søren Dinesen Østergaard","doi":"10.1111/acps.13791","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Clinical decision support systems based on machine learning (ML) models are emerging within psychiatry. To ensure their successful implementation, healthcare staff needs to trust these systems. Here, we investigated if providing staff with basic information about ML-based clinical decision support systems enhances their trust in them.</p><p><strong>Methods: </strong>We conducted a randomised survey experiment among staff in the Psychiatric Services of the Central Denmark Region. The participants were allocated to one of three arms, receiving different types of information: An intervention arm (receiving information on clinical decision-making supported by an ML model); an active control arm (receiving information on standard clinical decision process without ML support); and a blank control arm (no information). Subsequently, participants responded to various questions regarding their trust/distrust in ML-based clinical decision support systems. The effect of the intervention was assessed by pairwise comparisons between all randomization arms on sum scores of trust and distrust.</p><p><strong>Results: </strong>Among 2838 invitees, 780 completed the survey experiment. The intervention enhanced trust and diminished distrust in ML-based clinical decision support systems compared with the active control arm (Trust: mean difference = 5% [95% confidence interval (CI): 2%; 9%], p value < 0.001; Distrust: mean difference = -4% [-7%; -1%], p value = 0.042) and the blank control arm (Trust: mean difference = 5% [2%; 11%], p value = 0.003; Distrust: mean difference = -3% [-6%; -1%], p value = 0.021).</p><p><strong>Conclusion: </strong>Providing information on ML-based clinical decision support systems in hospital psychiatry may increase healthcare staff trust in such systems.</p>","PeriodicalId":108,"journal":{"name":"Acta Psychiatrica Scandinavica","volume":" ","pages":""},"PeriodicalIF":5.3000,"publicationDate":"2025-02-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Acta Psychiatrica Scandinavica","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1111/acps.13791","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHIATRY","Score":null,"Total":0}
引用次数: 0

Abstract

Background: Clinical decision support systems based on machine learning (ML) models are emerging within psychiatry. To ensure their successful implementation, healthcare staff needs to trust these systems. Here, we investigated if providing staff with basic information about ML-based clinical decision support systems enhances their trust in them.

Methods: We conducted a randomised survey experiment among staff in the Psychiatric Services of the Central Denmark Region. The participants were allocated to one of three arms, receiving different types of information: An intervention arm (receiving information on clinical decision-making supported by an ML model); an active control arm (receiving information on standard clinical decision process without ML support); and a blank control arm (no information). Subsequently, participants responded to various questions regarding their trust/distrust in ML-based clinical decision support systems. The effect of the intervention was assessed by pairwise comparisons between all randomization arms on sum scores of trust and distrust.

Results: Among 2838 invitees, 780 completed the survey experiment. The intervention enhanced trust and diminished distrust in ML-based clinical decision support systems compared with the active control arm (Trust: mean difference = 5% [95% confidence interval (CI): 2%; 9%], p value < 0.001; Distrust: mean difference = -4% [-7%; -1%], p value = 0.042) and the blank control arm (Trust: mean difference = 5% [2%; 11%], p value = 0.003; Distrust: mean difference = -3% [-6%; -1%], p value = 0.021).

Conclusion: Providing information on ML-based clinical decision support systems in hospital psychiatry may increase healthcare staff trust in such systems.

在精神科服务中接收基于机器学习的临床决策支持系统的信息增加了员工对这些系统的信任:一项随机调查实验。
背景:基于机器学习(ML)模型的临床决策支持系统正在精神病学领域兴起。为了确保其成功实施,医护人员需要信任这些系统。在此,我们调查了向员工提供基于ml的临床决策支持系统的基本信息是否会增强他们对该系统的信任。方法:我们对丹麦中部地区精神科服务的工作人员进行了随机调查实验。参与者被分配到三组中的一组,接收不同类型的信息:干预组(接收由ML模型支持的临床决策信息);主动控制臂(在没有ML支持的情况下接收标准临床决策过程的信息);空白的控制臂(没有信息)。随后,参与者回答了关于他们对基于ml的临床决策支持系统的信任/不信任的各种问题。通过对所有随机分组的信任和不信任总分的两两比较来评估干预的效果。结果:在2838名被邀请者中,780人完成了调查实验。与主动对照组相比,干预增强了基于ml的临床决策支持系统的信任,减少了不信任(信任:平均差值= 5%[95%置信区间(CI): 2%;结论:在医院精神病学中提供基于ml的临床决策支持系统的信息可以增加医护人员对该系统的信任。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Acta Psychiatrica Scandinavica
Acta Psychiatrica Scandinavica 医学-精神病学
CiteScore
11.20
自引率
3.00%
发文量
135
审稿时长
6-12 weeks
期刊介绍: Acta Psychiatrica Scandinavica acts as an international forum for the dissemination of information advancing the science and practice of psychiatry. In particular we focus on communicating frontline research to clinical psychiatrists and psychiatric researchers. Acta Psychiatrica Scandinavica has traditionally been and remains a journal focusing predominantly on clinical psychiatry, but translational psychiatry is a topic of growing importance to our readers. Therefore, the journal welcomes submission of manuscripts based on both clinical- and more translational (e.g. preclinical and epidemiological) research. When preparing manuscripts based on translational studies for submission to Acta Psychiatrica Scandinavica, the authors should place emphasis on the clinical significance of the research question and the findings. Manuscripts based solely on preclinical research (e.g. animal models) are normally not considered for publication in the Journal.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信