The fifty shades of black: about black box AI and explainability in healthcare.

Abstract

Artificial Intelligence (AI) is revolutionizing healthcare by enhancing patient care, diagnostics, workflows, and treatment personalization. The integration of AI in healthcare promises significant advancements and better patient outcomes. However, the lack of explainability in many AI models, known as 'black-box AI', raises concerns for patients, doctors, and developers. This issue, termed 'black box medicine', challenges the adoption of AI in healthcare. The demand for explainable AI has grown as AI systems become more complex. The absence of explanations in AI decisions, especially in critical situations like healthcare, has sparked debates and even suggestions to exclude black-box AI from healthcare provision. This article examines the impact and causes of unexplainable AI in healthcare, critically evaluates its performance, and proposes strategies to address this challenge.