A multilingual semantic search chatbot framework

Abstract

Chatbots are conversational agents which interact with users and simulate a human interaction. Companies use chatbots on their customer-facing sites to enhance user experience by answering questions about their products and directing users to relevant pages on the site. Existing Chatbots which are used for this purpose give responses based on pre-defined FAQs only. In this paper, we propose a framework for a chatbot which combines two approaches - retrieval from a knowledge base consisting of question answer pairs, combined with a natural language search mechanism which can scan through the paragraphs of text information. A feedback-based knowledge base update is implemented which provides continuous improvement in user experience. The framework achieves a result of 81.73 percent answer matching on SQuAD 1.1 and 69.21 percent answer matching on SQuAD 2.0. The framework also performs well on languages such as Spanish (67.32 percent answer match), Russian (61.43 percent answer match), Arabic (51.63 percent answer match) etc. by means of zero shot learning.