Predicting inappropriate source use from scores of language use, source comprehension, and organizational features: A study using generalized linear models

IF 4.2 1区文学 Q1 EDUCATION & EDUCATIONAL RESEARCH

Assessing Writing Pub Date : 2025-02-25 DOI:10.1016/j.asw.2025.100934

Kwangmin Lee , Ray J.T. Liao , I.-Chun Vera Hsiao , Junhee Park , Yafei Ye

{"title":"Predicting inappropriate source use from scores of language use, source comprehension, and organizational features: A study using generalized linear models","authors":"Kwangmin Lee , Ray J.T. Liao , I.-Chun Vera Hsiao , Junhee Park , Yafei Ye","doi":"10.1016/j.asw.2025.100934","DOIUrl":null,"url":null,"abstract":"<div><div>This paper examines the extent to which inappropriate source use – verbatim source use and patchwriting – can be predicted by scores of other textual features that are commonly evaluated in second/foreign language (L2) integrated writing assessment. 246 advanced-level English as a Foreign Language (EFL) test-takers enrolled in a Chinese higher education institution provided integrated essays that required both summary and argumentation. All the collected essays were rated by two experienced raters and checked for interrater reliability by way of generalizability theory. Then, a series of generalized linear models was compared to identify the best-fitting model that explained the relationship between the independent variables and inappropriate source use. Results indicated that the zero-inflated beta-binomial provided the best fit to the data, with approximately 43.67 % of the data estimated to be an extra zero. Next, parameter estimates of this model included (1) non-significant effects of <em>language use</em> and <em>source comprehension</em> on inappropriate source use and (2) a significantly negative effect of <em>organizational features</em> on the dependent variable. This suggests that focusing on <em>organizational features</em>, operationalized herein as organization, coherence, development of ideas, and authorial voice, can help L2 test-takers reduce reliance on inappropriate source use. Implications for research and practice are discussed.</div></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"64 ","pages":"Article 100934"},"PeriodicalIF":4.2000,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Assessing Writing","FirstCategoryId":"98","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1075293525000212","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}

引用次数: 0

Abstract

This paper examines the extent to which inappropriate source use – verbatim source use and patchwriting – can be predicted by scores of other textual features that are commonly evaluated in second/foreign language (L2) integrated writing assessment. 246 advanced-level English as a Foreign Language (EFL) test-takers enrolled in a Chinese higher education institution provided integrated essays that required both summary and argumentation. All the collected essays were rated by two experienced raters and checked for interrater reliability by way of generalizability theory. Then, a series of generalized linear models was compared to identify the best-fitting model that explained the relationship between the independent variables and inappropriate source use. Results indicated that the zero-inflated beta-binomial provided the best fit to the data, with approximately 43.67 % of the data estimated to be an extra zero. Next, parameter estimates of this model included (1) non-significant effects of language use and source comprehension on inappropriate source use and (2) a significantly negative effect of organizational features on the dependent variable. This suggests that focusing on organizational features, operationalized herein as organization, coherence, development of ideas, and authorial voice, can help L2 test-takers reduce reliance on inappropriate source use. Implications for research and practice are discussed.

查看原文本刊更多论文

从语言使用、来源理解和组织特征的分数预测不适当的来源使用：一项使用广义线性模型的研究

本文考察了在第二语言/外语（L2）综合写作评估中通常评估的其他文本特征的分数，可以在多大程度上预测不恰当的来源使用——逐字使用和拼凑。246名中国高等教育机构的高级英语作为外语（EFL）考生提供综合论文，要求总结和论证。所有收集的文章都由两位经验丰富的评分者进行评分，并通过概括性理论检验了评分者的信度。然后，比较了一系列广义线性模型，以确定最合适的模型来解释自变量与不适当的源使用之间的关系。结果表明，零膨胀的β -二项与数据的拟合效果最好，大约43.67 %的数据估计是一个额外的零。其次，该模型的参数估计包括(1)语言使用和源理解对不适当源使用的不显著影响，(2)组织特征对因变量的显著负向影响。这表明，关注组织特征（本文将其操作为组织性、连贯性、想法的发展和作者的声音）可以帮助二语考生减少对不适当的来源使用的依赖。讨论了对研究和实践的启示。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Assessing Writing Multiple-

CiteScore

6.00

自引率

17.90%

发文量

期刊介绍： Assessing Writing is a refereed international journal providing a forum for ideas, research and practice on the assessment of written language. Assessing Writing publishes articles, book reviews, conference reports, and academic exchanges concerning writing assessments of all kinds, including traditional (direct and standardised forms of) testing of writing, alternative performance assessments (such as portfolios), workplace sampling and classroom assessment. The journal focuses on all stages of the writing assessment process, including needs evaluation, assessment creation, implementation, and validation, and test development.