Generating Topic-Agnostic Conversations With LLMs

Abstract

Conversational systems are important applications of Artificial Intelligence, encompassing a wide variety of implementations, from rule-based systems to complex systems using Natural Language Processing, Deep Neural Networks, and Transformer Architectures. With the growth of these implementations, the quality of conversational data has become a concern. Many attempts have been made to generate such data, focusing primarily on topical conversations. This article presents a generalized framework moving from generating topical conversation towards topic-agnostic conversational data consisting of three Large Language Model instances. Two of these models interact with each other to generate the conversation, while the third one plays the role of a judge to keep the conversation going. The synthetic data created by the proposed method exhibits higher quality and lower toxicity than four of the existing datasets (AmazonQA, Daily Dialog, Open Subtitles, and HUMOD) in terms of six performance measures, namely Toxicity, Severe Toxicity, Obscene, Threat, Insult, and Identity Attack. Compared to other datasets, the performance analysis of the generated data shows the lowest measures in terms of mean and maximum values. Specifically, the percentages for Toxicity, Obscene, Threat, Insult, and Identity Attack are 0.27%, 0.04%, 0.02%, 0.05%, and 0.02%, respectively, while the corresponding maximum values are 90.13%, 29.65%, 3.42%, 67.16% and 0.69%. The generated dataset also shows the maximum concentration, with 99.74% of the data in the range of 0-10% toxicity with just a few outliers.