Assessing the effectiveness of large language models for intent detection in tourism chatbots: A comparative analysis and performance evaluation

Abstract

In recent years, the tourism industry has observed a significant transformation by integrating chatbots, which enable tourists to interact with various services using natural language. At the heart of each chatbot is a Natural Language Understanding (NLU) component, which processes natural language inputs through intent classification. This paper evaluates the performance of Large Language Models (LLMs) such as GPT, BERT, LLaMA, and RoBERTa in the intent classification task for tourism chatbots. Our study conducts a comparative analysis of various LLMs to determine their effectiveness in classifying user intents in tourism related interactions. We assess the models’ capabilities using a tourism-specific dataset labeled according to the “Six A” criteria for tourist destination analysis. The models are evaluated using performance metrics such as accuracy, precision, recall, and F1-score. The findings provide practical insights into developing efficient NLU components for tourism chatbots, enhancing their ability to understand and assist users effectively. This paper contributes to the field by offering a comprehensive performance evaluation of LLMs for NLU in tourism, guiding researchers and practitioners in building more responsive and accurate chatbots for the tourism industry.