Disciplinary variation of metadiscourse: A comparison of human-written and ChatGPT-generated English research article abstracts

Abstract

In order to identify more fundamental and subtler similarities and differences between human-written and ChatGPT-generated academic texts, and enhance the development and application of LLMs and understanding of human language, we use a self-built corpus and incorporate a bottom-up approach and statistical methods to compare metadiscourse variation across eight disciplines in human-written and ChatGPT-generated English research article abstracts. Results show that disciplinary variation of metadiscourse in human-written and ChatGPT-generated abstracts agrees in general but not in detail. Generally, in both types of abstracts, all disciplines use metadiscourse to fulfill three broad and eight specific discourse functions: Referring to text participants (Referring to writer, Referring to text), Describing text actions (Introducing, Arguing, Finding, Presenting), Describing text circumstances (Phoric marking, Code glossing), among which Referring to text participants and Introducing are prominent. Besides, disciplines in both types of abstracts exhibit the hard-soft discipline division in both frequencies and discourse functions, with metadiscourse markers and major discourse functions more prevalent in soft disciplines. Specifically, compared to disciplines of human-written abstracts, those of ChatGPT-generated abstracts differ more in frequencies but less in major discourse functions. The similarities and differences can be attributed to ChatGPT's working mechanism, training process, and limitation in accomplishing domain-specific tasks.