Artificial intelligent disobedience: Rethinking the agency of our artificial teammates

Abstract

The impressive advancements of AI in recent years are indisputable, and AI algorithms have already reached superhuman performance in many tasks. On the other hand, in most existing work on cooperative AI, artificial agents are bound to follow the instructions they are given by their human teammates and to comply with their users' expectations. This paper advocates for expanding the agency capabilities of AI teammates, enabling them to make genuine and unique contributions in human-AI teams. It presents a scale for AI agency levels and discusses the importance and inevitability of researching AI autonomy as an independent capability in cooperative AI, using a set of representative examples. The paper then presents how intelligent disobedience of artificial agents might look at each autonomy level. Finally, it outlines some initial boundaries that should be set when researching AI agency and its disobedience capabilities.