Optimizing chatbot responsiveness: Automated history context selector via three-way decision for multi-turn dialogue Large Language Models

Abstract

Enhancing the efficiency of chat models in multi-turn dialogue systems is a critical challenge in Artificial Intelligence. Multi-turn dialogues often span diverse topics, with irrelevant dialogue turns frequently degrading the quality of the model’s responses. This study addresses this challenge by proposing a novel method for the automated identification and selection of contextually relevant dialogue turns. Our approach introduces an Automated Relevance Labeling Pipeline, which leverages three-way decision and the K-Nearest Neighbors algorithm to automatically assign relevance labels by calculating the distance between dialogue turns and final responses. A Relevance Selector is trained on these labels, enabling it to accurately detect and prioritize relevant dialogue turns from the conversation history. The proposed method has been tested across various datasets demonstrating significant performance improvements over existing approaches that indiscriminately expand the entire conversation history. Notably, the integration of this method into existing chat models resulted in an increase in Recall rates by 4%–6% and a marked reduction in perplexity, approaching the accuracy of manually annotated data. The method’s zero-shot learning capabilities further underscore its generalizability applying to diverse conversational contexts without requiring additional fine-tuning. These results highlight the method’s potential to significantly enhance the performance of multi-turn dialogue systems.