It is not about autonomy: realigning the ethical debate on substitute judgement and AI preference predictors in healthcare.

Abstract

This article challenges two dominant assumptions in the current ethical debate over the use of algorithmic Personalised Patient Preference Predictors (P4) in substitute judgement for incapacitated patients. First, I question the belief that the autonomy of a patient who no longer has decision-making capacity can be meaningfully respected through a P4-empowered substitute judgement. Second, I critique the assumption that respect for autonomy can be reduced to merely satisfying a patient's individual treatment preferences. Both assumptions, I argue, are problematic: respect for autonomy cannot be equated with simply delivering the 'right' treatments, and expanding the normative scope of agency beyond first-person decisions creates issues for standard clinical decision-making. I suggest, instead, that the development of these algorithmic tools can be justified by achieving other moral goods, such as honouring a patient's unique identity or reducing surrogate decision-makers' burdens. This conclusion, I argue, should reshape the ethical debate around not just the future development and use of P4-like systems, but also on how substitute judgement is currently understood and justified in clinical medicine.