AI vs. human: A large-scale analysis of AI-generated fake reviews, human-generated fake reviews and authentic reviews

Abstract

The rise of fake reviews generated by artificial intelligence (AI) poses significant challenges for online platforms, retailers, and consumers, as advanced generative models like ChatGPT produce fake reviews. More seriously, unlike human-generated fake reviews, which exhibit psychological cues, AI-generated fake reviews lack deceptive intent and instead reflect algorithmic patterns that are increasingly difficult to detect. This study investigates the linguistic distinctions among AI-generated fake reviews, human-generated fake reviews, and authentic reviews by integrating the theoretical model of textual differences and AI characteristics. Analyzing 714,016 reviews through text analysis, entropy weight method and hypothesis testing, this research finds that AI-generated fake reviews demonstrate higher comprehensibility, lower levels of specificity, exaggeration, and negligence compared to human-generated fakes and authentic reviews. These findings challenge the existing paradigms in fake review research. This means that in traditional detection, the situations where there are AI-generated fake reviews that do not conform to rules should be given further consideration. Additionally, AI-generated fake reviews exhibit significantly higher mechanicalness and lower empathy than authentic reviews, providing actionable linguistic cues for identification. These findings highlight the need for revised detection methods that account for AI characteristics. Theoretically, it innovatively combines the textual differences model with AI capabilities to investigate AI-generated fake reviews, enriching the fake review detection literature. Practically, it offers e-commerce platforms and consumers enhanced detection metrics to improve algorithmic accuracy in identifying AI-generated deception.