Detection and classification of ChatGPT-generated content using deep transformer models.

Abstract

Introduction: The rapid advancement of AI, particularly artificial neural networks, has led to revolutionary breakthroughs and applications, such as text-generating tools and chatbots. However, this potent technology also introduces potential misuse and societal implications, including privacy violations, misinformation, and challenges to integrity and originality in academia. Several studies have attempted to distinguish and classify AI-generated textual content from human-authored work, but their performance remains questionable, particularly for AI models utilizing large language models like ChatGPT.

Methods: To address this issue, we compiled a dataset consisting of both human-written and AI-generated (ChatGPT) content. This dataset was then used to train and evaluate a range of machine learning and deep learning models under various training conditions. We assessed the efficacy of different models in detecting and classifying AI-generated content, with a particular focus on transformer-based architectures.

Results: Experimental results demonstrate that the proposed RoBERTa-based custom deep learning model achieved an F1-score of 0.992 and an accuracy of 0.991, followed by DistilBERT, which yielded an F1-score of 0.988 and an accuracy of 0.988. These results indicate exceptional performance in detecting and classifying AI-generated content.

Discussion: Our findings establish a robust baseline for the detection and classification of AI-generated textual content. This work marks a significant step toward mitigating the potential misuse of AI-powered text generation tools by providing a reliable approach for distinguishing between human and AI-generated text. Future research could explore the generalizability of these models across different AI-generated content sources and address evolving challenges in AI text detection.