Heuristic Evaluation of Public Service Chatbots

Abstract

In recent years, chatbots have been adopted in business contexts and also for public services at a growing rate. Chatbots provide dialogue interfaces combining visual elements with natural conversation. Good Conversational Design in this context covers the topics of Natural-Language Processing (NLP) and Dialogue Management (DM). Few attention has been paid to the usability evaluation of conversational interfaces (Höhn & Bongard-Blanchy, 2021). The present paper builds upon the work by Höhn & Bongard-Blanchy by applying their framework of conversational heuristics to evaluate a set of public service chatbots operated in the federal state of Schleswig-Holstein. Thus, for each public service chatbot, a usability score is established and typical characteristics of public service chatbots in general are summarized. We discuss results by comparing the overall scores, weaknesses and strengths of each chatbot. In addition, we reflect on our experience in the application of the framework as well as highlight possible optimization potentials. Concludingly, this paper collects UX recommendations for public service chatbots.