In Algorithms We Trust? Stakeholder Perspectives on Algorithmic and AI-Based Survival Calculators.

Abstract

Background: The integration of digital algorithms and artificial intelligence (AI) into healthcare has enabled personalized medicine, e.g., prognostic calculators that predict survival/mortality estimates and patient risk stratification. Alongside these advancements, understanding patients' trust in and perspectives on the use of these tools in their healthcare is essential.

Methods: Interviews were conducted with advanced heart failure patients (and caregivers) who received algorithmic personalized survival estimates (PSEs) in the context of education and decision making about left-ventricular assist device (LVAD) therapy as part of their clinical care. We examine stakeholder trust considerations for their algorithmic-based PSEs received during LVAD education and the potential impact of AI-generated PSEs on trust in PSEs.

Results: Rationales for patient trust in their received, algorithmic PSEs fell into three categories - relational (e.g., physician trusts algorithm's output so patient does), epistemic (e.g., algorithm accuracy and performance), and personal belief-based (e.g., religious beliefs about algorithmic vs. God's predictive power). Relational trust considerations were mostly absent from trust considerations for hypothetical AI-generated PSEs, and despite acknowledgement of potential performance benefits (greater predictive power) of AI-based tools, responses were dominated by personal belief-based trust considerations rooted in inaccurate understandings of the application of AI in the healthcare context (e.g., AI as "the Terminator").

Conclusions: While algorithmic PSEs benefit from epistemic and relational trust considerations, AI-based PSEs may face distinct barriers rooted in misconceptions about AI's capabilities or application. Combating such misconceptions about AI may require a combination of approaches, like furthering AI education, physician oversight for AI systems utilized in their patients' care, and more conscientious framing and language regarding what AI is and can do. As AI continues to integrate into medical care and decision-making, these findings underscore the importance of acknowledging and addressing personal and relational dimensions of patient trust in AI systems in addition to epistemic or performance aspects.