以人为中心的人工智能用于紧急心脏护理：用PROLIFERATE_AI评估RAPIDx人工智能

IF 3.7 2区医学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

International Journal of Medical Informatics Pub Date : 2025-01-28 DOI:10.1016/j.ijmedinf.2025.105810

Maria Alejandra Pinero de Plaza , Kristina Lambrakis , Fernando Marmolejo-Ramos , Alline Beleigoli , Mandy Archibald , Lalit Yadav , Penelope McMillan , Robyn Clark , Michael Lawless , Erin Morton , Jeroen Hendriks , Alison Kitson , Renuka Visvanathan , Derek P. Chew , Carlos Javier Barrera Causil

{"title":"以人为中心的人工智能用于紧急心脏护理：用PROLIFERATE_AI评估RAPIDx人工智能","authors":"Maria Alejandra Pinero de Plaza , Kristina Lambrakis , Fernando Marmolejo-Ramos , Alline Beleigoli , Mandy Archibald , Lalit Yadav , Penelope McMillan , Robyn Clark , Michael Lawless , Erin Morton , Jeroen Hendriks , Alison Kitson , Renuka Visvanathan , Derek P. Chew , Carlos Javier Barrera Causil","doi":"10.1016/j.ijmedinf.2025.105810","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Chest pain diagnosis in emergency care is hindered by overlapping cardiac and non-cardiac symptoms, causing diagnostic uncertainty. Artificial Intelligence, such as RAPIDx AI, aims to enhance accuracy through clinical and biochemical data integration, but its adoption relies on addressing usability, explainability, and seamless workflow integration without disrupting care.</div></div><div><h3>Objective</h3><div>Evaluate RAPIDx AI’s integration into clinical workflows, address usability barriers, and optimise its adoption in emergencies.</div></div><div><h3>Methods</h3><div>The PROLIFERATE_AI framework was implemented across 12 EDs (July 2022–January 2024) with 39 participants: 15 experts co-designed a survey via Expert Knowledge Elicitation (EKE), applied to 24 ED clinicians to assess RAPIDx AI usability and adoption. Bayesian inference, using priors, estimated comprehension, emotional engagement, usage, and preference, while Monte Carlo simulations quantified uncertainty and variability, generating posterior means and 95% bootstrapped confidence intervals. Qualitative thematic analysis identified barriers and optimisation needs, with data triangulated through the PROLIFERATE_AI scoring system to rate RAPIDx AI’s performance by user roles and demographics.</div></div><div><h3>Results</h3><div>Registrars exhibited the highest comprehension (median: 0.466, 95 % CI: 0.41–0.51) and preference (median: 0.458, 95 % CI: 0.41–0.48), while residents/interns scored the lowest in comprehension (median: 0.198, 95 % CI: 0.17–0.26) and emotional engagement (median: 0.112, 95 % CI: 0.09–0.14). Registered nurses showed strong emotional engagement (median: 0.379, 95 % CI: 0.35–0.45). Novice users faced usability and workflow integration barriers, while experienced clinicians suggested automation and streamlined workflows. RAPIDx AI scored “Good Impact,” excelling with trained users but requiring targeted refinements for novices.</div></div><div><h3>Conclusion</h3><div>RAPIDx AI enhances diagnostic accuracy and efficiency for experienced users, but usability challenges for novices highlight the need for targeted training and interface refinements. The PROLIFERATE_AI framework offers a robust methodology for evaluating and scaling AI solutions, addressing the evolving needs of sociotechnical systems.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"196 ","pages":"Article 105810"},"PeriodicalIF":3.7000,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Human-centred AI for emergency cardiac care: Evaluating RAPIDx AI with PROLIFERATE_AI\",\"authors\":\"Maria Alejandra Pinero de Plaza , Kristina Lambrakis , Fernando Marmolejo-Ramos , Alline Beleigoli , Mandy Archibald , Lalit Yadav , Penelope McMillan , Robyn Clark , Michael Lawless , Erin Morton , Jeroen Hendriks , Alison Kitson , Renuka Visvanathan , Derek P. Chew , Carlos Javier Barrera Causil\",\"doi\":\"10.1016/j.ijmedinf.2025.105810\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Background</h3><div>Chest pain diagnosis in emergency care is hindered by overlapping cardiac and non-cardiac symptoms, causing diagnostic uncertainty. Artificial Intelligence, such as RAPIDx AI, aims to enhance accuracy through clinical and biochemical data integration, but its adoption relies on addressing usability, explainability, and seamless workflow integration without disrupting care.</div></div><div><h3>Objective</h3><div>Evaluate RAPIDx AI’s integration into clinical workflows, address usability barriers, and optimise its adoption in emergencies.</div></div><div><h3>Methods</h3><div>The PROLIFERATE_AI framework was implemented across 12 EDs (July 2022–January 2024) with 39 participants: 15 experts co-designed a survey via Expert Knowledge Elicitation (EKE), applied to 24 ED clinicians to assess RAPIDx AI usability and adoption. Bayesian inference, using priors, estimated comprehension, emotional engagement, usage, and preference, while Monte Carlo simulations quantified uncertainty and variability, generating posterior means and 95% bootstrapped confidence intervals. Qualitative thematic analysis identified barriers and optimisation needs, with data triangulated through the PROLIFERATE_AI scoring system to rate RAPIDx AI’s performance by user roles and demographics.</div></div><div><h3>Results</h3><div>Registrars exhibited the highest comprehension (median: 0.466, 95 % CI: 0.41–0.51) and preference (median: 0.458, 95 % CI: 0.41–0.48), while residents/interns scored the lowest in comprehension (median: 0.198, 95 % CI: 0.17–0.26) and emotional engagement (median: 0.112, 95 % CI: 0.09–0.14). Registered nurses showed strong emotional engagement (median: 0.379, 95 % CI: 0.35–0.45). Novice users faced usability and workflow integration barriers, while experienced clinicians suggested automation and streamlined workflows. RAPIDx AI scored “Good Impact,” excelling with trained users but requiring targeted refinements for novices.</div></div><div><h3>Conclusion</h3><div>RAPIDx AI enhances diagnostic accuracy and efficiency for experienced users, but usability challenges for novices highlight the need for targeted training and interface refinements. The PROLIFERATE_AI framework offers a robust methodology for evaluating and scaling AI solutions, addressing the evolving needs of sociotechnical systems.</div></div>\",\"PeriodicalId\":54950,\"journal\":{\"name\":\"International Journal of Medical Informatics\",\"volume\":\"196 \",\"pages\":\"Article 105810\"},\"PeriodicalIF\":3.7000,\"publicationDate\":\"2025-01-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Medical Informatics\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1386505625000279\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Medical Informatics","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1386505625000279","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

摘要

背景：胸痛诊断在急诊护理受阻重叠的心脏和非心脏症状，造成诊断的不确定性。人工智能，如RAPIDx AI，旨在通过临床和生化数据集成来提高准确性，但其采用依赖于在不中断护理的情况下解决可用性、可解释性和无缝工作流程集成问题。目的：评估RAPIDx AI与临床工作流程的整合，解决可用性障碍，并优化其在紧急情况下的采用。方法：在12个急诊科实施了PROLIFERATE_AI框架（2022年7月至2024年1月），共有39名参与者：15名专家通过专家知识启发（EKE）共同设计了一项调查，应用于24名急诊科临床医生，以评估RAPIDx AI的可用性和采用情况。贝叶斯推理，使用先验，估计理解，情感投入，使用和偏好，而蒙特卡罗模拟量化不确定性和可变性，生成后验均值和95%自举置信区间。定性专题分析确定了障碍和优化需求，并通过PROLIFERATE_AI评分系统对数据进行三角测量，根据用户角色和人口统计数据对RAPIDx AI的性能进行评分。结果：登记员表现出最高的理解（中位数：0.466,95% CI: 0.41-0.51）和偏好（中位数：0.458,95% CI: 0.41-0.48），而住院医生/实习生在理解（中位数：0.198,95% CI: 0.17-0.26）和情感投入（中位数：0.112,95% CI: 0.09-0.14）方面得分最低。注册护士表现出较强的情绪投入（中位数：0.379,95% CI: 0.35-0.45）。新手用户面临可用性和工作流集成障碍，而经验丰富的临床医生建议自动化和简化工作流程。RAPIDx AI获得了“良好的影响”，在训练有素的用户中表现出色，但需要针对新手进行有针对性的改进。结论：RAPIDx AI为经验丰富的用户提高了诊断的准确性和效率，但对于新手的可用性挑战突出了有针对性的培训和界面改进的必要性。PROLIFERATE_AI框架为评估和扩展AI解决方案提供了一个强大的方法，解决了社会技术系统不断发展的需求。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Human-centred AI for emergency cardiac care: Evaluating RAPIDx AI with PROLIFERATE_AI

Background

Chest pain diagnosis in emergency care is hindered by overlapping cardiac and non-cardiac symptoms, causing diagnostic uncertainty. Artificial Intelligence, such as RAPIDx AI, aims to enhance accuracy through clinical and biochemical data integration, but its adoption relies on addressing usability, explainability, and seamless workflow integration without disrupting care.

Objective

Evaluate RAPIDx AI’s integration into clinical workflows, address usability barriers, and optimise its adoption in emergencies.

Methods

The PROLIFERATE_AI framework was implemented across 12 EDs (July 2022–January 2024) with 39 participants: 15 experts co-designed a survey via Expert Knowledge Elicitation (EKE), applied to 24 ED clinicians to assess RAPIDx AI usability and adoption. Bayesian inference, using priors, estimated comprehension, emotional engagement, usage, and preference, while Monte Carlo simulations quantified uncertainty and variability, generating posterior means and 95% bootstrapped confidence intervals. Qualitative thematic analysis identified barriers and optimisation needs, with data triangulated through the PROLIFERATE_AI scoring system to rate RAPIDx AI’s performance by user roles and demographics.

Results

Registrars exhibited the highest comprehension (median: 0.466, 95 % CI: 0.41–0.51) and preference (median: 0.458, 95 % CI: 0.41–0.48), while residents/interns scored the lowest in comprehension (median: 0.198, 95 % CI: 0.17–0.26) and emotional engagement (median: 0.112, 95 % CI: 0.09–0.14). Registered nurses showed strong emotional engagement (median: 0.379, 95 % CI: 0.35–0.45). Novice users faced usability and workflow integration barriers, while experienced clinicians suggested automation and streamlined workflows. RAPIDx AI scored “Good Impact,” excelling with trained users but requiring targeted refinements for novices.

Conclusion

RAPIDx AI enhances diagnostic accuracy and efficiency for experienced users, but usability challenges for novices highlight the need for targeted training and interface refinements. The PROLIFERATE_AI framework offers a robust methodology for evaluating and scaling AI solutions, addressing the evolving needs of sociotechnical systems.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

International Journal of Medical Informatics 医学-计算机：信息系统

CiteScore

8.90

自引率

4.10%

发文量

217

审稿时长

42 days

期刊介绍： International Journal of Medical Informatics provides an international medium for dissemination of original results and interpretative reviews concerning the field of medical informatics. The Journal emphasizes the evaluation of systems in healthcare settings. The scope of journal covers: Information systems, including national or international registration systems, hospital information systems, departmental and/or physician''s office systems, document handling systems, electronic medical record systems, standardization, systems integration etc.; Computer-aided medical decision support systems using heuristic, algorithmic and/or statistical methods as exemplified in decision theory, protocol development, artificial intelligence, etc. Educational computer based programs pertaining to medical informatics or medicine in general; Organizational, economic, social, clinical impact, ethical and cost-benefit aspects of IT applications in health care.