Maximizing machines: Does cognitive alignment impact delegation to artificial intelligence agents in a simulated triage scenario

Abstract

Within the study of human-artificial Intelligence (AI) teaming, a key question has been when and why a human will delegate a task to an AI. Increasingly researchers are looking at the role of AI alignment in delegation and trust. Such perspectives emphasize the importance of value alignment, meaning that the AI should prioritize the same values as a human. What is missing, however, is an examination of alignment not just focused on the values that drive an AI, but the process in which it conducts that task. That is, the importance of an AI that uses a decision-making process, not just outcomes, that matches a human. To address this gap in the AI, alignment and trust literature, we explore the effect of alignment to the cognitive trait maximization on AI delegation. Using a modified version of the Least-Worst Uncertain Choice Inventory for Emergency Responders (LUCIFER), we explored how alignment on maximization impacted delegation in a mock-triage scenario across three distinct studies that explored alignment (study 1), the effect of feedback (study 2), and decision-making stakes (study 3). Overall, trust in AI was the most robust and consistent predictor of delegation behavior, reaching significance in all three studies for delegation percentage. The effect of alignment, by contrast, was context-dependent. It did not reach significance as a main effect in any study but produced some exploratory interaction effects with alignment to maximization and satisficing traits in Study 3. The implications of this for contemporary discussions of AI, delegation and decision-making are discussed.