Do I have a say?: Using conversational agents to re-imagine human-machine autonomy

Abstract

With human-centered AI gaining traction, needs for algorithmic transparency, explainability, empathy and ethical considerations in artificial agents are forming core research issues. However, with intelligent agents getting increasingly independent and human-like, there is an increase in perceived threat to human autonomy. Will this perceived threat eventually become an actual threat where humans lose control over their own goals, decisions and actions? With this provocation paper, we want to urge researchers working on human-agent interactions and conversational agents (CAs) to explicitly consider the impact of intelligent agents on human autonomy. We present arguments and highlight the critical need to focus on human autonomy when interacting with CAs by presenting some core research considerations.