Assessing Writing最新文献_第8页

Establishing analytic score profiles for large-scale L2 writing assessment: The case of the CET-4 writing test 为大规模 L2 写作评估建立分析分数档案：以 CET-4 写作测试为例

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-02-19 DOI: 10.1016/j.asw.2024.100826

Shaoyan Zou , Xun Yan , Jason Fan

{"title":"Establishing analytic score profiles for large-scale L2 writing assessment: The case of the CET-4 writing test","authors":"Shaoyan Zou , Xun Yan , Jason Fan","doi":"10.1016/j.asw.2024.100826","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100826","url":null,"abstract":"<div><p>This study addresses a critical need in large-scale L2 writing assessment by emphasizing the significance of tailoring assessments to specific teaching and learning contexts. Focusing on the CET-4 writing test in China, the research unfolded in two phases. In Phase I, an empirically-developed analytic rating scale designed for the CET-4 writing test was rigorously validated. Twenty-one raters used this scale to rate 30 essays, and Many-Facets Rasch Model (MFRM) analysis was performed on the rating data. The outcomes demonstrate the scale’s robustness in effectively differentiating examinees’ writing performance, ensuring consistency among raters, and mitigating rater variation at both individual and group level. Phase II extends the research scope by applying the validated scale to score 142 CET-4 writing scripts. Utilizing Hierarchical and K-Means cluster analyses, this phase unveils three distinct score profiles. These findings are significant for both the CET-4 writing test and other L2 large-scale writing assessment. Theoretically, this study introduces a perspective that aims to enhance our understanding of learners’ performance in large-scale L2 writing assessment. Methodologically, this study presents a framework that integrates the validation of the rating scale with the identification of distinct score clusters, thus aiming to provide a more detailed solution for tailoring assessments to specific learning contexts.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"60 ","pages":"Article 100826"},"PeriodicalIF":3.9,"publicationDate":"2024-02-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139907854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Validity of automated essay scores for elementary-age English language learners: Evidence of bias? 针对小学英语学习者的自动作文评分的有效性：偏见的证据？

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-02-17 DOI: 10.1016/j.asw.2024.100815

Joshua Wilson , Yue Huang

{"title":"Validity of automated essay scores for elementary-age English language learners: Evidence of bias?","authors":"Joshua Wilson , Yue Huang","doi":"10.1016/j.asw.2024.100815","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100815","url":null,"abstract":"<div><p>Given increased prevalence of automated writing evaluation (AWE) systems in classroom settings, more research is needed to explore the potential for bias in automated scores with respect to English language learners (ELLs). Thus, this research study investigated and compared the predictive validity of automated and human scoring methods for elementary-age English ELLs on a writing test designed for ELLs and a state writing test designed for the general population. This study focused on the MI Write AWE system and sampled 2829 students comprising ELLs and non-ELLs in Grades 3–5. Results of multilevel regression analyses and simple slopes estimation indicated that, for ELLs, the automated MI Write score had similar predictive validity to the human score for both writing tests. However, automated and human scores for ELLs were less closely related to the state writing test score than scores for non-ELL students. Findings suggest that MI Write’s automated scoring was not uniquely biased relative to human scoring but does reproduce the same biases evident with human scoring. Implications and directions for future research are discussed.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"60 ","pages":"Article 100815"},"PeriodicalIF":3.9,"publicationDate":"2024-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139898550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-02-17 DOI: 10.1016/j.asw.2024.100828

Jianling Zhan , Chuyi Zhou

引用次数: 0

Exploring the effects of task difficulty and learner variables on performance on picture description writing tasks 探索任务难度和学习者变量对图片描述写作任务成绩的影响

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-02-16 DOI: 10.1016/j.asw.2024.100827

Khaled Barkaoui

{"title":"Exploring the effects of task difficulty and learner variables on performance on picture description writing tasks","authors":"Khaled Barkaoui","doi":"10.1016/j.asw.2024.100827","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100827","url":null,"abstract":"<div><p>Picture description (PD) tasks are widely used in second language (L2) teaching, research, and assessment to elicit and/or evaluate L2 learners’ writing performance. However, there is little research on the effects of the characteristics of such tasks on L2 writing performance. This study aimed to examine the effects of task difficulty and learner variables on the linguistic characteristics of responses to the 2020 version of the Duolingo English Test (DET) PD writing tasks. The written responses of 1439 test takers from four first language (L1) backgrounds and at different levels of L2 proficiency to 335 DET PD tasks at two levels of difficulty were analyzed and compared in terms of measures of fluency, accuracy, and complexity using various computer programs. The findings indicated that task difficulty did not affect writing performance significantly; high-proficiency learners tended to perform better than did their low-proficiency counterparts; responses receiving higher scores tended to be longer and more accurate; and learner L1 was significantly associated with writing grades and linguistic features. The findings and their implications for the DET PD task and the DET validity argument are discussed.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"60 ","pages":"Article 100827"},"PeriodicalIF":3.9,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139743711","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Assessing video game narratives: Implications for the assessment of multimodal literacy in ESP 评估电子游戏叙事：对 ESP 中多模态素养评估的启示

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-02-02 DOI: 10.1016/j.asw.2024.100809

Vicent Beltrán-Palanques

{"title":"Assessing video game narratives: Implications for the assessment of multimodal literacy in ESP","authors":"Vicent Beltrán-Palanques","doi":"10.1016/j.asw.2024.100809","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100809","url":null,"abstract":"<div><p>Research into the contribution of multimodality to language learning is gaining momentum. While most studies pave the way for new understandings of language teaching and learning, there is an increasing demand for comprehensive assessment practices, particularly within higher education contexts. A few studies have emphasized the importance of reflecting on and establishing criteria for the assessment of multimodal literacy. This is necessary to understand students’ contributions in detail and to provide them with effective support in developing their multimodal skills. This study discusses the assessment of multimodal writing in English for Specific Purposes (ESP) contexts. It presents the design of an analytical tool for assessing multimodal texts and provided an example of its application. This tool covers assessment categories such as language use, content expression, interpersonal meaning, multimodality, and creativity and originality. As an example, we focus on the multimodal writing of a video game narrative, a genre that requires the integration of multiple modes of communication to convey meaning more effectively. Finally, this study offers pedagogical insights into the assessment of multimodal literacy in ESP.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"60 ","pages":"Article 100809"},"PeriodicalIF":3.9,"publicationDate":"2024-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1075293524000023/pdfft?md5=eaba395b3bc5d3f49c5ee81b441c065b&pid=1-s2.0-S1075293524000023-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139674270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A comparison between input modalities and languages in source-based multilingual argumentative writing 基于源语言的多语言议论文写作中输入模式和语言的比较

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-01-31 DOI: 10.1016/j.asw.2024.100813

Roberto Arias-Hermoso, Ainara Imaz Agirre, Eneritz Garro Larrañaga

{"title":"A comparison between input modalities and languages in source-based multilingual argumentative writing","authors":"Roberto Arias-Hermoso, Ainara Imaz Agirre, Eneritz Garro Larrañaga","doi":"10.1016/j.asw.2024.100813","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100813","url":null,"abstract":"<div><p><span>This paper examines secondary education<span> students’ multilingual writing based on two input modalities to understand the influence of input on argumentation and sourcing strategies. Participants produced texts in </span></span>Basque, Spanish and English based on a video or a text, and texts were analysed to explore their production of argumentation elements and sourcing strategies. Differences were found between input modalities in the use of data and rebuttals, and in copying. Across-language differences were also found in the use of data and counterarguments, and in the use of original ideas and paraphrasing. Additionally, complex argumentation elements elicited more original ideas than simple ones. Findings suggest that different writing sub-processes might be activated when composing from different sources and that argumentation and sourcing might be transferable across languages. These results may have important implications for educators in multilingual programs who aim to support their students in acquiring academic writing skills in multiple languages.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"60 ","pages":"Article 100813"},"PeriodicalIF":3.9,"publicationDate":"2024-01-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139652749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Shaping Writing Grades：Collocation and Writing Context Effects, McCallum, Lee, Philip Durrant, Cambridge Elements in Corpus Linguistics, Cambridge University Press, Cambridge (2022), p. 88, ISBN: 9781009074445

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-01-01 DOI: 10.1016/j.asw.2023.100807

Xiaoyi Yang, Yuan Ping

引用次数: 0

Noun phrase complexity and second language Chinese proficiency: An analysis of narratives by Korean learners of Chinese 名词短语复杂性与第二语言汉语水平：韩国汉语学习者的叙述分析

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-01-01 DOI: 10.1016/j.asw.2024.100810

Jifeng Wu , Xiaofei Lu

{"title":"Noun phrase complexity and second language Chinese proficiency: An analysis of narratives by Korean learners of Chinese","authors":"Jifeng Wu , Xiaofei Lu","doi":"10.1016/j.asw.2024.100810","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100810","url":null,"abstract":"<div><p>This study examined differences in the overall noun phrase (NP) complexity and the use of specific types of NP modifiers among Korean learners of Chinese as a second language (CSL) at different proficiency levels and first language (L1) Chinese speakers. Our data consisted of a corpus of 134 narratives produced by 103 Korean CSL learners (33 beginner, 37 intermediate, and 33 advanced) and 31 L1 Chinese speakers on the same topic. Each narrative was manually analyzed using seven measures of overall NP complexity and 26 measures based on specific types of NP modifiers. Two measures of overall NP complexity (i.e., complex NP ratio and total length of all complex premodifiers) significantly discriminated all learner proficiency levels. The frequency of several types of NP modifiers consistently decreased or increased from lower to higher proficiency levels, while that of several others showed cross-proficiency fluctuations. Advanced learners demonstrated comparable levels of overall NP complexity as well as similar frequency of usage of specific types of NP modifiers as L1 Chinese speakers, with the exception that the latter used significantly more multiple modifiers. Our findings have useful implications for L2 Chinese writing syntactic complexity research and L2 Chinese writing pedagogy.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"59 ","pages":"Article 100810"},"PeriodicalIF":3.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1075293524000035/pdfft?md5=78fe5dfebdb0aa9f7137263e1abc7b2c&pid=1-s2.0-S1075293524000035-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139652720","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

第二语言或外语写作诊断》，Ari Huhta、Claudia Harsch、Dmitri Leontjev、Lea Nieminen，Routledge，纽约（2024 年），Xii+Pp.334

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-01-01 DOI: 10.1016/j.asw.2024.100814

Kai Zhao

引用次数: 0

Assessing metacognition-based student feedback literacy for academic writing 评估基于元认知的学生学术写作反馈素养

IF 3.9 1区文学

Assessing Writing Pub Date : 2024-01-01 DOI: 10.1016/j.asw.2024.100811

Mark Feng Teng , Maggie Ma

{"title":"Assessing metacognition-based student feedback literacy for academic writing","authors":"Mark Feng Teng , Maggie Ma","doi":"10.1016/j.asw.2024.100811","DOIUrl":"https://doi.org/10.1016/j.asw.2024.100811","url":null,"abstract":"<div><p>Student feedback literacy is essential to writing in a foreign language context. Feedback is a process through which learners become more familiar with their work quality and integrate constructive criticism to produce stronger written work. Implicit in this process is the assumption that learners should not passively receive information but actively seek, generate, process, and use feedback to apply new knowledge in current or subsequent writing tasks. Learners’ metacognitive awareness and skills may influence their participation in feedback-related activities that inform the monitoring and control of their writing process and determine their writing performance. However, little attention has been given to a systematic investigation of metacognition in feedback literacy. The present study, drawing upon a self-regulatory perspective, bridges this gap by fulfilling two purposes: (a) to measure a scale on metacognition-based student feedback literacy; and (b) to delineate the predictive effects of different components of the scale on academic writing performance. The results provided evidence for metacognitive awareness and skills in student feedback literacy and showed the predictive effects of the scale, e.g., feedback-related strategies in participation (FRSP), motivation (M), feedback-related monitoring strategies (FRMS), and strategy knowledge (SK) on EFL learners’ performance in academic writing. Relevant implications were also discussed.</p></div>","PeriodicalId":46865,"journal":{"name":"Assessing Writing","volume":"59 ","pages":"Article 100811"},"PeriodicalIF":3.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1075293524000047/pdfft?md5=bab5f2c98aefcabe1eebf93cef0f6bd3&pid=1-s2.0-S1075293524000047-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139674633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0