Let XAI generate reliability metadata, not medical explanations

Abstract

As AI becomes increasingly embedded in medical practice, the call for explainability – commonly framed as eXplainable AI (XAI) – has grown, especially under regulatory pressures. However, conventional XAI approaches misunderstand clinical decision-making by focusing on post-hoc explanations rather than actionable cues. This letter argues that to calibrate trust in AI recommendations, physicians’ primary need is not for conventional post-hoc explanations, but for “reliability metadata”: a set of both marginal and instance-specific indicators that facilitate the assessment of the reliability of each individual advice given. We propose shifting the focus from generating static explanations to providing actionable cues – such as calibrated confidence scores, out-of-distribution alerts, and relevant reference cases – that support adaptive reliance and mitigate automation bias. By reframing XAI as eXtended and eXplorable AI, we emphasize interaction, uncertainty transparency, and clinical relevance over explanations per se. This perspective encourages AI design that aligns with real-world medical cognition, promotes reflective engagement, and supports safer, more effective decision-making.