Chatbot Evaluation as Knowledge Application: a Case Study of PT ABC

Abstract

ELISA chatbot acts as an automated answering machine to users' problem in PT ABC. Implemented in May 2018, it succeeded to provide a faster response time than offline customer service. However, it still has issues in delivering accurate and appropriate responses. One of the reasons is the lack of system control and evaluation. This paper aims to evaluate how ELISA chatbot, as a knowledge application, benefits the company using DeLone & McLean model of information system as a reference. A survey was conducted, and SEM-PLS was utilized to process the data. The result shows that information quality, service quality, and intention to use significantly affect user satisfaction in ELISA. Suggestions for ELISA improvement includes machine learning technology utilization and nonfunctional improvements.