Explaining AI weaknesses improves human–AI performance in a dynamic control task

AI-based decision support is increasingly implemented to support operators in dynamic control tasks. While these systems continuously improve, to truly achieve human–system synergy, one must also study humans’ system understanding and behavior. Accordingly, we investigated the impact of explainability instructions regarding a specific system weakness on performance and trust in two experiments (with higher task demands in Experiment 2). Participants performed a dynamic control task with support from either an explainable AI (XAI, information on a system weakness), a non-explainable AI (nonXAI, no information on system weakness), or without support (manual, only in Experiment 2). Results show that participants with XAI support outperformed those in the nonXAI group, particularly in situations where the AI actually erred. Notably, informing users of system weaknesses did not affect trust once they had interacted with the system. In addition, Experiment 2 showed the general benefit of decision support over working manually under higher task demands. These findings suggest that AI support can enhance performance in complex tasks and that providing information on potential system weaknesses aids in managing system errors and resource allocation without compromising trust.