Stack More LLM’s: Efficient Detection of Machine-Generated Texts via Perplexity Approximation

Abstract

The development of large language models (LLMs) is currently receiving a great amount of interest, but an update of text generation methods should entail a continuous update of methods for detecting machine-generated texts. Earlier, it has been highlighted that values of perplexity and log-probability are able to capture a measure of the difference between artificial and human-written texts. Using this observation, we define a new criterion based on these two values to judge whether a passage is generated from a given LLM. In this paper, we propose a novel efficient method that enables the detection of machine-generated fragments using an approximation of the LLM perplexity value based on pre-collected statistical language models. Approximation lends a hand in achieving high performance and quality metrics also on fragments from weights-closed LLMs. A large number of pre-collected statistical dictionaries results in an increased generalisation ability and the possibility to cover text sequences from the wild. Such approach is easy to update by only adding a new dictionary with latest model text outputs. The presented method has a high performance and achieves quality with an average of 94% recall in detecting generated fragments among texts from various open-source LLMs. In addition, the method is able to perform in milliseconds, which outperforms state-of-the-art models by a factor of thousands.