A content-controlled monolingual comparable corpus approach to comparing learner and proficient argumentative writing

This mixed-methods study approaches the differences between learner and proficient argumentative writing by building a content-controlled monolingual comparable corpus (CCMCC) that contains learner-teacher sample pairs of the same semantic content. Twenty-seven learner samples were collected from 27 Chinese university students who each wrote on one topic from the second writing task of IELTS Academic. To generate content-controlled teacher samples, an experienced teacher revised or rewrote each learner sample after confirming the ideas learners intended to express through individual and face-to-face communication with each learner. Then, a native speaker checked the language of the teacher samples. In data analysis, each learner-teacher sample pair was analyzed using Coh-Metrix to generate statistics in 45 indices under text length, syntax, lexicon, and cohesion, after which a shortlist of indices of both statistically and practically significant differences was identified. Qualitatively, the researchers identified the differences through side-by-side comparisons of sample pairs and coded the important patterns in the differences to explore their underlying reasons. This approach generated different quantitative results from previous corpus-based comparative writing studies, such as the ineffectiveness of cohesion indices to distinguish learner and proficient writing. Qualitative analysis further revealed noteworthy findings including the lack of concision in learner writing and learners’ unfamiliarity with using prepositional phrases to express actions. The advantages, limitations and implications of this approach are discussed.