Assessing Writing最新文献_第5页

How reliable and valid is peer evaluation in adolescents’ L2 argumentative writing? 同伴评价在青少年第二语言议论文写作中的可靠性和有效性如何？

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-11-29 DOI: 10.1016/j.asw.2025.100992

Albert W. Li , Steve Graham

{"title":"How reliable and valid is peer evaluation in adolescents’ L2 argumentative writing?","authors":"Albert W. Li , Steve Graham","doi":"10.1016/j.asw.2025.100992","DOIUrl":"10.1016/j.asw.2025.100992","url":null,"abstract":"<div><div>Peer evaluation is widely recognized for its educational benefits; however, its reliability and validity, particularly among adolescent second-language (L2) writers at the early stages of English language and literacy development, remain insufficiently explored. This explanatory sequential mixed-methods study investigated the reliability and validity of peer evaluation in English argumentative writing among 35 Grade 10 and 37 Grade 12 students from a public high school in Beijing, China. Twelve of the participating students (six at each grade) were interviewed about the validity, reliability, and value of peer evaluation. The findings indicated that peer evaluations demonstrated high levels of reliability and validity, with peer-assessed writing scores closely aligning with inter-teacher assessments. Notably, variations were observed among Grade 10 students, particularly in the evaluation of lower-order writing skills, such as grammar and vocabulary, which exhibited reduced validity. These results underscore the potential of peer evaluation in assessing higher-order content-level writing across varying levels of L2 English writing proficiency. The study also highlights areas where adolescent L2 writers may require additional support to enhance the effectiveness of peer evaluation practices in English argumentative writing. Implications for improving English argumentative writing instruction and refining peer evaluation strategies in high school L2 English classrooms are discussed.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100992"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145624789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Extracting interpretable writing traits from a large language model 从大型语言模型中提取可解释的写作特征

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2026-01-19 DOI: 10.1016/j.asw.2025.101011

Paul Deane, Andrew Hoang

{"title":"Extracting interpretable writing traits from a large language model","authors":"Paul Deane, Andrew Hoang","doi":"10.1016/j.asw.2025.101011","DOIUrl":"10.1016/j.asw.2025.101011","url":null,"abstract":"<div><div>Large language models (LLMs) are increasingly used to support automated writing evaluation (AWE), both for purposes of scoring and feedback. However, LLMs present challenges to interpretability, making it hard to evaluate the construct validity of scoring and feedback models. BIOT (best interpretable orthogonal transformations) is a new method of analysis that makes dimensions of an embedding interpretable by aligning them with external predictors. It was originally developed to improve the interpretability of multidimensional scaling models. However, This paper shows that BIOT can be used to align LLM embeddings with an interpretable writing trait model developed using multidimensional analysis of classical NLP features to measure latent dimensions of writing style and writing quality. This makes it possible to determine whether an AWE model built using an LLM is aligned with known (and construct-relevant) dimensions of textual variation, supporting construct validity. Specifically, we examine the alignment between the hidden layers of deBERTA, a small LLM that has been shown to be useful for a variety of natural language processing applications, and a writing trait model developed through factor analysis of classical features used in existing AWE models. Specific dimensions of transformed deBERTA layers are strongly correlated with these classical factors. When the transformation matrix derived using BIOT is applied to token vectors, it is also possible to visualize which tokens in the original text contributed to high or low scores on a specific dimension.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 101011"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146037269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Assessing EFL students’ GenAI-assisted writing: Teachers’ pains, perceptions and practices 评估英语学生的genai辅助写作：教师的痛苦、认知和实践

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-12-03 DOI: 10.1016/j.asw.2025.100996

Xiaoxiao Chen , Xiaojun Pi

{"title":"Assessing EFL students’ GenAI-assisted writing: Teachers’ pains, perceptions and practices","authors":"Xiaoxiao Chen , Xiaojun Pi","doi":"10.1016/j.asw.2025.100996","DOIUrl":"10.1016/j.asw.2025.100996","url":null,"abstract":"<div><div>As generative artificial intelligence (GenAI) becomes increasingly embedded in EFL students’ writing practices, it has raised profound challenges to traditional assessment systems, calling into question the validity of established rubrics, the reliability of teacher judgments, and the authenticity of student performance. Despite this, limited research has explored how EFL teachers perceive and respond to GenAI-assisted writing. This qualitative case study investigates four EFL writing teachers’ assessment behaviors and reasoning across two stages of blind and informed evaluations of GenAI-assisted student texts. Drawing on think-aloud protocols and interviews, the study reveals that while the EFL writing teachers struggled to identify GenAI-generated content without prior knowledge, some exhibited noticeable adjustments in their scoring behaviors and assessment criteria once informed of GenAI involvement, whereas others remained relatively consistent. These changes were closely tied to their GenAI knowledge and experience and underlying pedagogical beliefs. The findings highlight divergent assessment strategies, ranging from product-oriented scoring to critical scrutiny of student-AI interaction, and underscore the salience of teacher agency in shaping responses to technological disruption. By uncovering the teachers’ cognitive processes, dilemmas, and reflective practices, the study contributes to a deeper understanding of assessment literacy in AI-mediated learning environments and offers practical implications for rubric adaptation, teacher professional development, and policy support.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100996"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145684370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Assessing the effects of task complexity on cognitive demands in L2 writing 评估任务复杂性对二语写作认知需求的影响

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-12-04 DOI: 10.1016/j.asw.2025.100998

Na Tao , Ying Wang

{"title":"Assessing the effects of task complexity on cognitive demands in L2 writing","authors":"Na Tao , Ying Wang","doi":"10.1016/j.asw.2025.100998","DOIUrl":"10.1016/j.asw.2025.100998","url":null,"abstract":"<div><div>The assessment of task-generated cognitive demands has been receiving increasing attention in task complexity research. However, scant attention has been paid to assessing cognitive demands when task complexity is manipulated along both resource-directing and resource-dispersing dimensions. To address this gap, the present study aimed to investigate the relative effects of reasoning demands and prior knowledge on cognitive demands in L2 writing. Eighty-eight EFL students completed two letter-writing tasks with varying reasoning demands under one of two conditions, that is, either with prior knowledge available or without prior knowledge available. Cognitive demands were assessed by the post-task questionnaire, the dual-task method and the open-ended questions. The results revealed that reasoning demands and prior knowledge were strong determinants of cognitive demands, which provided empirical evidence for Robinson’s Cognition Hypothesis. Moreover, the post-task questionnaire, the dual-task method and open-ended questions were found to assess distinct aspects of cognitive demands, which highlighted the importance of data triangulation in exploring task complexity effects. The study provides language teachers and assessors with implications for task design and implementation.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100998"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145684367","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Development of a Genre Adherence Rubric (GAR) for applied linguistics research articles 应用语言学研究论文体裁依附性标准的建立

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-11-15 DOI: 10.1016/j.asw.2025.100991

Mahsa Alinasab , Javad Gholami , Zhila Mohammadnia

{"title":"Development of a Genre Adherence Rubric (GAR) for applied linguistics research articles","authors":"Mahsa Alinasab , Javad Gholami , Zhila Mohammadnia","doi":"10.1016/j.asw.2025.100991","DOIUrl":"10.1016/j.asw.2025.100991","url":null,"abstract":"<div><div>This study reports the development of a four-descriptor Genre Adherence Rubric (GAR) for Research Articles (RAs) based on two pilot studies. To this end, we designed, implemented, and assessed a genre-oriented RA writing course in a master's program in applied linguistics. The instructional package contained knowledge-giving and hands-on materials and tasks on moves/steps, their sequencing, and linguistic features in RA sections. The participants were asked to revise their first draft RAs following the course. We defined, developed, and piloted the GAR, which primarily consisted of move obligation, optionality, and sequencing, and used it to rate the original and revised RAs. The first pilot and scorer feedback showed that language needs to be included as an additional descriptor. In the second pilot study, implementing the four-prong GAR yielded meaningful differences in another set of revised RAs. As a novel attempt to rate RAs and similar scholarly writings through genre lenses and apart from opening new avenues for research, the GAR presented in this paper warrants further confirmation or modification. Given the ever-growing importance of scholarly writing and publishing, the findings have tenable implications for journal editors, publishers, and academic writing instructors to adopt or adapt GAR-like RA appraisal scales.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100991"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145521191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Generative artificial intelligence for automated essay scoring: Exploring teacher agency through an ecological perspective 自动作文评分的生成式人工智能：从生态角度探索教师代理

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-11-11 DOI: 10.1016/j.asw.2025.100990

Jessie S. Barrot

{"title":"Generative artificial intelligence for automated essay scoring: Exploring teacher agency through an ecological perspective","authors":"Jessie S. Barrot","doi":"10.1016/j.asw.2025.100990","DOIUrl":"10.1016/j.asw.2025.100990","url":null,"abstract":"<div><div>Generative artificial intelligence (AI) is increasingly used in writing assessment, particularly for automated essay scoring (AES) and for generating formative feedback within automated writing evaluation (AWE). While AI-driven AES enhances efficiency and consistency, concerns regarding accuracy, bias, and ethical implications raise critical questions about its role in assessment. This paper examines the impact of generative AI on teacher agency through an ecological perspective, which considers agency as shaped by personal, institutional, and sociocultural factors. The analysis highlights the need for teachers to critically mediate AI-generated scores and feedback to align them with pedagogical goals, ensuring AI functions as an assistive tool rather than a determinant of assessment outcomes. Although AI can streamline assessment, over-reliance risks diminishing teachers’ evaluative expertise and reinforcing biases embedded in AI systems. Ethical concerns, including transparency, data privacy, and fairness, further complicate its adoption. To address these challenges, this paper proposes a framework for responsible AI integration that prioritizes bias mitigation, data security, and teacher-driven decision-making. The discussion concludes with pedagogical implications and directions for future research on AI-assisted writing assessment.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100990"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145486179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Assessing the effects of explicit coherence instruction on EFL students’ integrated writing performance 评估显性连贯教学对英语学生综合写作成绩的影响

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2026-01-30 DOI: 10.1016/j.asw.2026.101019

Xi Li , Mo Chen

{"title":"Assessing the effects of explicit coherence instruction on EFL students’ integrated writing performance","authors":"Xi Li , Mo Chen","doi":"10.1016/j.asw.2026.101019","DOIUrl":"10.1016/j.asw.2026.101019","url":null,"abstract":"<div><div>As a key attribute of effective writing, coherence remains challenging to teach in language classrooms, with traditional writing instruction frequently overlooking coherence in favor of discrete, rule-based features. This mixed-methods study investigates the effectiveness of explicit coherence instruction on English-as-a-Foreign-Language (EFL) students’ performance on integrated writing tasks. The study employed a controlled experimental design with 64 upper-intermediate-level undergraduate students at a Chinese university, drawing on Hasan’s Cohesive Harmony theory as the theoretical framework. Half of the participants (n = 32) in the experimental group received explicit instruction on coherence with a focus on cohesive chains and cohesive devices in integrated writing, while the control group (n = 32) received standard paraphrasing instruction. Quantitative analysis revealed that the experimental group showed significant improvements in coherence scores and multiple cohesive chain measures. Qualitative discourse analysis of six students’ writing samples from the experimental group demonstrated varying levels of improvement in writing coherence, with high-performing students showing better use of identity chains and pronoun references. The findings revealed that explicit instruction on coherence significantly improved students’ performance in creating coherent integrated writing, particularly through the development of cohesive chains and appropriate use of cohesive devices. This study underscores the pedagogical value of teaching coherence to enhance writing quality and provides concrete strategies for developing more effective teaching approaches for integrated writing tasks in EFL contexts.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 101019"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146077315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The effects of online resource use on L2 learners’ computer-mediated writing processes and written products 网络资源使用对二语学习者计算机辅助写作过程和写作成果的影响

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-12-02 DOI: 10.1016/j.asw.2025.100994

Honglan Wang , Jookyoung Jung

{"title":"The effects of online resource use on L2 learners’ computer-mediated writing processes and written products","authors":"Honglan Wang , Jookyoung Jung","doi":"10.1016/j.asw.2025.100994","DOIUrl":"10.1016/j.asw.2025.100994","url":null,"abstract":"<div><div>While previous studies on online resource use in L2 writing have focused on the overall writing quality, limited attention has been paid to its effects on linguistic complexity and real-time writing processes. Addressing this gap, the present study explored how online resource use influences both the processes and products of L2 writing. Forty-nine intermediate L2 learners completed two computer-mediated argumentative writing tasks, either with or without the use of online resources. Writing behaviors were captured via keystroke logging and screen recording, and analyzed for search activity, fluency, pausing, and revision quantity. Cognitive processes were examined through stimulated recall interviews, and written products were evaluated for both quality and linguistic complexity. The results showed that participants spent an average of 14 % of task time using online resources, with considerable individual variation. Mixed-effects modeling revealed that resource use facilitated the production of more sophisticated words, with marginal influence on writing quality or syntactic complexity. Resource use was also associated with longer between-word pauses, fewer within-word pauses, and reduced revisions. These findings highlight the potential of online resource use to enhance the authenticity of L2 writing assessment tasks without compromising test validity, while encouraging the use of more advanced vocabulary in writing.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"67 ","pages":"Article 100994"},"PeriodicalIF":5.5,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145684368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Volume 67 editorial 第67卷社论

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2026-01-29 DOI: 10.1016/j.asw.2026.101016

Martin East , David Slomp

引用次数: 0

Verb-centric or balanced?: An NLP-based assessment of word class contributions to L2 writing proficiency 动词中心还是平衡？基于nlp的词类对第二语言写作能力贡献的评估

IF 5.5 1区文学

Assessing Writing Pub Date : 2026-01-01 Epub Date: 2025-12-06 DOI: 10.1016/j.asw.2025.100997

Hyunwoo Kim , Haerim Hwang

引用次数: 0