Assessing Writing最新文献_第4页

Developing students’ feedback literacy in disciplinary academic writing through generative artificial intelligence 通过生成式人工智能培养学生在学科学术写作中的反馈素养

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-02-14 DOI: 10.1016/j.asw.2026.101030

Jianda Liu , Zihao Shi , Wanqing Li

{"title":"Developing students’ feedback literacy in disciplinary academic writing through generative artificial intelligence","authors":"Jianda Liu , Zihao Shi , Wanqing Li","doi":"10.1016/j.asw.2026.101030","DOIUrl":"10.1016/j.asw.2026.101030","url":null,"abstract":"<div><div>Recent research has explored how generative artificial intelligence (GenAI) may support peer feedback practices and foster students’ feedback literacy (SFL) in L2 writing, yet its distinctive role remains under-examined, particularly in disciplinary academic writing. Addressing this gap, this mixed-methods study investigated the effects of GenAI feedback on Chinese EFL students’ feedback literacy in disciplinary writing contexts. Changes in SFL were compared between a control group receiving peer feedback and an experimental group receiving GenAI feedback before and after an intervention, with qualitative and text data used to interpret quantitative results of the GenAI feedback. The experimental group showed significant gains across all dimensions of SFL. Qualitative findings further indicated improved ability to interpret, evaluate, and act on feedback. However, learners reported sustained cognitive load when engaging with GenAI feedback, and limited disciplinary knowledge and unfamiliarity with academic conventions continued to generate negative emotions and constrain help-seeking. Pedagogical implications for integrating GenAI feedback into disciplinary writing instruction are discussed.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101030"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174390","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating GPT ratings of EFL writing: A scoping review 评估英语写作的GPT评级：范围审查

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-04-06 DOI: 10.1016/j.asw.2026.101044

Yi Chen

{"title":"Evaluating GPT ratings of EFL writing: A scoping review","authors":"Yi Chen","doi":"10.1016/j.asw.2026.101044","DOIUrl":"10.1016/j.asw.2026.101044","url":null,"abstract":"<div><div>The rise of large language models (LLMs), exemplified by GPT, has opened new possibilities for automated essay scoring (AES) in L2 education. Over the past three years, a growing number of studies have investigated GPT’s potential as a rater of English as a foreign language (EFL) writing. However, with no synthesis existing, the literature remains relatively fragmented. To address this gap, this scoping review analyzed 26 identified studies in terms of their research designs, evaluation foci, reported findings, and summative evaluations. Collectively, these studies addressed three core aspects of the evaluation inference in a full validity argument—accuracy, consistency, and fairness, and presented a cautiously positive view of GPT’s performance in rating EFL essays. Preliminary insights include GPT-4 and GPT-4o’s superiority over standard GPT-3.5 in accuracy and consistency, the promise of few-shot learning prompts, and GPT’s tendency to score more severely in language-related dimensions. The review also identified some methodological limitations across the literature, and highlighted key areas for further investigation. By providing a structured overview of this emerging field for the first time, this scoping review offers guidance for future research and for L2 educators considering the use of GPT models in EFL writing assessment.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101044"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147649897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

L2 learners’ engagement with AI-generated feedback on writing 第二语言学习者使用人工智能生成的写作反馈

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-02-12 DOI: 10.1016/j.asw.2026.101020

Xinyu Ma , Cong Zhang , Icy Lee

{"title":"L2 learners’ engagement with AI-generated feedback on writing","authors":"Xinyu Ma , Cong Zhang , Icy Lee","doi":"10.1016/j.asw.2026.101020","DOIUrl":"10.1016/j.asw.2026.101020","url":null,"abstract":"<div><div>While feedback has gained considerable attention in second language (L2) writing, little is known about generative AI-generated feedback, especially how students engage with it during the revision process. Drawing on qualitative data from multiple textual sources and in-depth interviews, the study explored the cognitive, behavioral, and affective engagement of six Chinese EFL students with AI feedback generated by ChatGPT. Cognitively, students demonstrated an improved understanding of AI feedback through sustained interactions with AI and engaged in cognitive and metacognitive operations including setting goals, retrieving prior knowledge, and evaluating. Behaviorally, students were more receptive to local feedback than global feedback. Affectively, most students displayed critical attitudes toward AI feedback, except for one student who expressed skeptical attitudes. They also experienced a range of emotional responses and fluctuations when dealing with feedback. They employed revision strategies such as planning the revision sequence and consulting external resources. Additionally, the three dimensions of engagement were found to be interconnected with each other, with learner agency playing a vital role in the tripartite framework of engagement. The study contributes to the understanding of applying AI feedback to L2 writing, highlighting its potential to foster student engagement and support writing development.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101020"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Associations of adolescents’ argumentative writing scores and growth when evaluated by different human raters and artificial intelligence models 不同人类评分者和人工智能模型评估青少年议论文写作分数与成长的关系

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-01-23 DOI: 10.1016/j.asw.2026.101015

Deborah K. Reed , Sterett Mercer

{"title":"Associations of adolescents’ argumentative writing scores and growth when evaluated by different human raters and artificial intelligence models","authors":"Deborah K. Reed , Sterett Mercer","doi":"10.1016/j.asw.2026.101015","DOIUrl":"10.1016/j.asw.2026.101015","url":null,"abstract":"<div><div>Having accurate and timely information on students’ argumentative writing skills is necessary to foster their improvement. Therefore, this study explored the associations among two different human rater types (researchers and teachers) and four different artificial intelligence (AI) models when using an analytic rubric to score eighth and ninth graders' argumentative writing gathered at two time points. Results revealed that, at each time point, AI models and teacher raters generally assigned higher scores than researchers. There tended to be moderate agreement of AI model scores with researcher scores that generally exceeded the agreement of teacher scores with researchers. However, there also tended to be greater agreement among the scores from AI models compared to the agreement of AI scores with either human rater type. There were some differences by writing trait and by AI model, suggesting ChatGPT-4o scores had the strongest association with researcher scores particularly for the task and development traits. When examining growth over time, the teacher and researcher scores demonstrated positive growth, whereas the AI scores generally suggested students declined. Again, the AI models had more concordance to each other than to human estimations of growth, but there was slightly more agreement in the estimates when comparing researchers to AI than to teachers. Implications for data use and interpretation are discussed.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101015"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146025937","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

From spelling to content: The influence of spelling quality on text assessment 从拼写到内容：拼写质量对课文评价的影响

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-01-21 DOI: 10.1016/j.asw.2026.101014

Frederike Strahl , Jörg Kilian , Jens Möller

{"title":"From spelling to content: The influence of spelling quality on text assessment","authors":"Frederike Strahl , Jörg Kilian , Jens Möller","doi":"10.1016/j.asw.2026.101014","DOIUrl":"10.1016/j.asw.2026.101014","url":null,"abstract":"<div><h3>Background</h3><div>Text assessments by different teachers can lack objectivity, with some text characteristics influencing the evaluation of others.</div></div><div><h3>Aims</h3><div>This research explored the halo effect of spelling quality on first language text assessment and investigated whether a prompt could reduce this effect.</div></div><div><h3>Samples</h3><div>Study 1 involved 134 pre-service teachers and 83 in-service teachers. Study 2 included 130 in-service teachers, who were divided into three groups.</div></div><div><h3>Methods</h3><div>In Study 1, participants assessed six student texts with varying overall and spelling quality on a holistic, content, style, and linguistic accuracy scale. Study 2 replicated Study 1, but additionally involved two prompting groups, emphasizing independent assessments of text characteristics.</div></div><div><h3>Results</h3><div>In both studies, texts with better spelling quality received higher ratings on spelling independent and spelling dependent scales, demonstrating successful manipulation and halo effects of spelling quality on content and style assessments. Study 2 showed robust halo effects in first language assessment, even after prompting.</div></div><div><h3>Conclusion</h3><div>The studies showed halo effects of spelling quality in text assessment, which remained stable even when prompted to reduce them. Future research should test more detailed interventions to reduce halo effects in text assessment.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101014"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146025936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AI tools vs human in english narrative writing scoring: A comparative study of scoring agreement among human raters, DeepSeek and ChatGPT in exam context 人工智能工具与人类英语叙事写作评分：人类评分者、DeepSeek和ChatGPT在考试环境下评分一致性的比较研究

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-04-06 DOI: 10.1016/j.asw.2026.101040

Yan Liang , Yueh Yea Lo

{"title":"AI tools vs human in english narrative writing scoring: A comparative study of scoring agreement among human raters, DeepSeek and ChatGPT in exam context","authors":"Yan Liang , Yueh Yea Lo","doi":"10.1016/j.asw.2026.101040","DOIUrl":"10.1016/j.asw.2026.101040","url":null,"abstract":"<div><div>A growing body of literature is examining the agreement between artificial intelligence (AI) and humans in scoring. However, empirical findings remain inconclusive, ranging from a strong correlation to a large difference in the scores assigned by the two media. In response, this mixed-methods study compared scoring agreement and evaluation rationales across three sources, including ChatGPT-4, DeepSeek-R1 and human ratings from a city-level standarised English narrative writing examination (n = 212). Quantitatively, the findings revealed strong relative agreement between the two AI models, while only moderate alignment was observed between each AI tool and the human ratings. Qualitative analyses of a focus group interview and scoring rationales indicated that these discrepancies reflect systematic differences in evaluation orientation. Human raters demonstrated more flexibility and sensitivity to contextual and presentation-related factors in evaluating writing quality, whereas AI tools prioritised linguistic accuracy and structural regularity and remained unaffected by surface features such as handwriting or erasures. Rather than suggesting simple scoring errors, the results reveal systematic differences between human and AI judgment, with implications for the principled integration of AI into high-stakes assessment.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101040"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147649898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Conceptual, rhetorical and linguistic transformations: Assessing L2 literature review writing using simulated tasks 概念、修辞和语言转换：用模拟任务评估第二语言文献综述写作

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-01-20 DOI: 10.1016/j.asw.2026.101013

Haiying Feng , Lawrence Jun Zhang , Kexin Li

{"title":"Conceptual, rhetorical and linguistic transformations: Assessing L2 literature review writing using simulated tasks","authors":"Haiying Feng , Lawrence Jun Zhang , Kexin Li","doi":"10.1016/j.asw.2026.101013","DOIUrl":"10.1016/j.asw.2026.101013","url":null,"abstract":"<div><div>This research addresses a critical gap in assessing literature review (LR) writing among L2 learner writers in terms of knowledge transformation. By integrating three strands of literature—genre analysis, source-based writing, and discourse synthesis—this study develops a framework that conceptualizes knowledge transformation along three dimensions: conceptual, rhetorical, and linguistic. It examines transforming performance of 125 Chinese students across bachelor, master, and doctoral levels through two simulated LR writing tasks, and explores the influence of factors including language proficiency and prior genre knowledge on their performance. Furthermore, it reveals the intricate interacting relationship among six transforming indicators (i.e., content units included, accuracy of information representation, extent of source integration, academic critiques and niche construction, verbatim copying, and lexico-grammatical errors). The results of textual and statistical analysis offer pedagogical implications, including enhancing compare-and-contrast practice, cultivating awareness of niche construction, and reducing L2 writers’ cognitive load in linguistic transformation.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101013"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145996550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Generative artificial intelligence for automated writing evaluation: A systematic review of trends, efficacy, and challenges 自动写作评估的生成人工智能：趋势、功效和挑战的系统回顾

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-03-30 DOI: 10.1016/j.asw.2026.101041

Shadi I. Abudalfa , Jessie S. Barrot

{"title":"Generative artificial intelligence for automated writing evaluation: A systematic review of trends, efficacy, and challenges","authors":"Shadi I. Abudalfa , Jessie S. Barrot","doi":"10.1016/j.asw.2026.101041","DOIUrl":"10.1016/j.asw.2026.101041","url":null,"abstract":"<div><div>Generative artificial intelligence (GenAI) has been increasingly integrated into education, but its application in automated writing evaluation (AWE) remains underexplored. The absence of a comprehensive synthesis leaves unresolved questions about their effectiveness, reliability, and ethical implications in high-stakes contexts. This study addresses that gap through a systematic review of 96 empirical works on GenAI in AWE, consolidating evidence on efficacy, challenges, and pedagogical value. Following PRISMA guidelines and guided by the CIMO framework, the review combined quantitative descriptive mapping with qualitative thematic synthesis to evaluate interventions, mechanisms, and outcomes. The analysis shows that GenAI systems perform efficiently in grammar correction, coherence, and rubric-based tasks while remaining inconsistent in assessing higher-order skills such as argumentation and creativity. Findings also reveal conditional effectiveness, with outcomes shaped by context, task design, and prompting strategies, alongside mixed impacts on learner motivation and teacher workload. Despite aligning with human raters on surface-level accuracy, unresolved concerns regarding validity, equity, and academic integrity limit their readiness for high-stakes assessment. The review concludes that GenAI in AWE represents both promise and risk, requiring careful integration and human oversight. These insights carry significant implications for theory, pedagogy, and methodology.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101041"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147600054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Task complexity, collaborative writing, and learner engagement: Examining second language learners’ writing performance 任务复杂性、协作写作和学习者参与：第二语言学习者写作表现的检验

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-04-01 Epub Date: 2026-03-31 DOI: 10.1016/j.asw.2026.101042

Syed Muhammad Mujtaba , Tiefu Zhang , Natalia Sletova

{"title":"Task complexity, collaborative writing, and learner engagement: Examining second language learners’ writing performance","authors":"Syed Muhammad Mujtaba , Tiefu Zhang , Natalia Sletova","doi":"10.1016/j.asw.2026.101042","DOIUrl":"10.1016/j.asw.2026.101042","url":null,"abstract":"<div><div>Collaboration and learner engagement have been found to positively influence second language (L2) writing development. However, few studies have examined how tasks of differing cognitive demands, such as narrative (simpler) versus argumentative (more complex), affect collaborative writing performance and learner engagement. To address this gap, two intact undergraduate classes in an EAP course were assigned to either the individual or the collaborative writing condition. Participants completed narrative and argumentative tasks, with writing performance measured through syntactic complexity, accuracy, lexical complexity, and fluency (CALF), as well as cognitive and behavioural engagement. The collaborative condition consistently outperformed the individual condition in accuracy and fluency across tasks. Within both conditions, the complex task was associated with higher fluency, while accuracy was higher in the simple task. Task complexity had no significant effect on syntactic or lexical complexity. Additionally, task complexity did not significantly alter cognitive or behavioural engagement in the individual condition; however, it significantly increased behavioural engagement, measured as time on task, in the collaborative condition. The findings support the pedagogical value of integrating collaborative writing tasks with different cognitive demands in L2 instruction.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"68 ","pages":"Article 101042"},"PeriodicalIF":5.5,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147600053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The relation between linguistic accuracy and scoring of Swedish EFL students’ writing during a high-stakes exam 瑞典英语学生高风险考试写作中语言准确性与得分的关系

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-12-05 DOI: 10.1016/j.asw.2025.100995

Christian Holmberg Sjöling

{"title":"The relation between linguistic accuracy and scoring of Swedish EFL students’ writing during a high-stakes exam","authors":"Christian Holmberg Sjöling","doi":"10.1016/j.asw.2025.100995","DOIUrl":"10.1016/j.asw.2025.100995","url":null,"abstract":"<div><div>This paper examines the effect of linguistic accuracy (e.g., the lack of form, grammatical, and lexical errors) on scoring during the high-stakes national test of English in Swedish upper secondary school. Teachers are expected to score their own students’ texts with the help of assessment instructions containing benchmark texts (i.e., texts representing different score bands). The assessment instructions and the score bands provided to guide scoring are not explicit about how accuracy should influence scores. Two research questions were answered: As measured by ordinal regression, to what extent does linguistic accuracy predict rater scores? Do the texts scored by teachers reflect the graded example texts in terms of how linguistic accuracy predicts scores? The results revealed, amongst other things, that overall frequency of errors in texts significantly predicted scores as the model explained approximately 58 % of the variance in the outcome variable according to Nagelkerke’s pseudo R-squared. Accuracy also had a similar effect on scores in texts rated by teachers as in the benchmark texts. In relation to the findings, it was concluded that accuracy may have more of an impact on scores than constructs that are more explicit components of the score bands such as lexical complexity.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100995"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145684369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0