A comparative genre analysis of AI-generated and scholar-written abstracts for English review articles in international journals

Abstract

There has been growing interest in the performance and efficiency of ChatGPT in generating academic texts. However, little empirical research has been conducted on its performance in producing review article abstracts. This study adopts the genre analysis approach to investigate the rhetorical moves of review article abstracts in hard and soft science disciplines based on two self-compiled corpora, respectively including 160 scholar-written abstracts from four high-impact international journals, and 160 abstracts generated by ChatGPT, with an aim to reveal the similarities and differences between human-written and AI-generated English review article abstracts. The results show significant differences between human-written and ChatGPT-generated abstracts, first in the frequency of three out of the five moves, and then in the sequential order of moves, with each type of abstracts demonstrating a preference for move sequence patterns as well as obligatory and optional elements. The two types of abstracts differ significantly in the frequency of move embedding, but share the same embedding combination patterns. These findings may deepen our understanding of ChatGPT's capabilities and limitations in generating academic texts across different disciplines, help improve the generative AI system, then highlight the complex relationship among the structure of academic abstracts, discipline cultures and genre knowledge.