A Question Answering Chatbot for Gastric Cancer Patients After Curative Gastrectomy: Development and Evaluation of User Experience and Performance.

Abstract

Postoperative gastric cancer patients have many questions about managing their daily lives with various symptoms and discomfort. This study aimed to develop a knowledge-based question answering (QA) chatbot for their self-management and to evaluate the user experience and performance of the chatbot. To support the chatbot's natural language processing, we analyzed QA texts from an online self-help group, clinical guidelines, and refined frequently asked questions related to gastric cancer. We developed a named entity classification with seven superconcepts, 4544 subconcepts, and 1415 synonyms. We also developed a knowledge base by linking the users' classified question intents with the experts' answers and knowledge resources, including 677 question intents and scripts with standard QA pairs and similar question phrases. A chatbot called "GastricFAQ" was built, reflecting the question topics of the named entity classification and QA pairs of the knowledge base. User experience evaluation (N = 56) revealed the highest mean score for usefulness (4.41/5.00), with all other items rated 4.00 or higher, except desirability (3.85/5.00). The chatbot's accuracy, precision, recall, and F score ratings were 85.2%, 87.6%, 96.8%, and 92.0%, respectively, with immediate answers. GastricFAQ could be provided as one option to obtain immediate information with relatively high accuracy for postoperative gastric cancer patients.