Cognitive challenges in human-AI collaboration: Investigating the path towards productive delegation

Abstract

We study how humans make decisions when they collaborate with an artificial intelligence (AI): each instance of a classification task could be classified by themselves or by the AI. Experimental results suggest that humans and AI who work together can outperform the superior AI when it works alone. However, this only occurred when the AI delegated work to humans, not when humans delegated work to the AI. The AI profited, even from working with low-performing subjects, but humans did not delegate well. This bad delegation performance cannot be explained with algorithm aversion. On the contrary, subjects tried to follow a provided delegation strategy diligently and appeared to appreciate the AI support. However, human results suffered due to a lack of metaknowledge. They were not able to assess their own capabilities correctly, which in turn leads to poor delegation decisions. In contrast to reluctance to use AI, lacking metaknowledge
is an unconscious trait. It limits fundamentally how well human decision makers can collaborate with AI and other algorithms when there is no explicit performance feedback. The results have implications for the future of work, the design of human-AI collaborative environments and education in the digital age.