Trust in the system and human autonomy in customer service chatbots

Abstract

When Artificial Intelligence systems are not explained clearly to users, it can negatively affect their interactions and compromise their perceptions of a brand. When designing and developing conversational agents that deal with the client, it is crucial to consider that they are a service and follow human-centered Artificial Intelligence (HCAI) approaches. This study discusses two HCAI frameworks, relate them to trust in the system and human autonomy and define how these guidelines could be met in customer service chatbot. A survey was conducted to determine if users' views about their interactions with chatbots aligned with the recommended guidelines and how this affected their senses mentioned above. The analysis of the responses indicates that those human-centered Artificial Intelligence approaches still need to be prioritized or even met in customer service chatbot development. Users have reported unpleasant experiences with such services, leading to a decrease in their trust and autonomy.