Text Authenticity in Listening Assessment: Can Item Writers Be Trained to Produce Authentic-sounding Texts?

IF 1.4 2区 文学 0 LANGUAGE & LINGUISTICS
Olena Rossi, Tineke Brunfaut
{"title":"Text Authenticity in Listening Assessment: Can Item Writers Be Trained to Produce Authentic-sounding Texts?","authors":"Olena Rossi, Tineke Brunfaut","doi":"10.1080/15434303.2021.1895162","DOIUrl":null,"url":null,"abstract":"ABSTRACT A long-standing debate in the testing of listening concerns the authenticity of the listening input. On the one hand, listening texts produced by item writers often lack spoken language characteristics. On the other hand, real-life recordings are often too context-specific to stand alone, or not suitable for item generation. In this study, we explored the effectiveness of an existing item-writing training course to produce authentic-sounding listening texts within the constraints of test specifications. Twenty-five trainees took an online item-writing course including training on creating authentic-sounding listening texts. Prior to and after the course, they developed a listening task. The resulting listening texts were judged on authenticity by three professional item reviewers and analysed linguistically by the researchers. Additionally, we interviewed the trainees following each item writing event and analysed their online discussions from during the course. Statistical comparison of the pre-and post-course authenticity scores revealed a positive effect of the training on item-writers’ ability to produce authentic-sounding listening texts, while the linguistic analysis demonstrated that the texts produced after the training contained more instances of spoken language. The interviews and discussions revealed that item writers’ awareness of spoken language features and their text production techniques influenced their ability to develop authentic-sounding texts.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":"18 1","pages":"398 - 418"},"PeriodicalIF":1.4000,"publicationDate":"2021-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/15434303.2021.1895162","citationCount":"6","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Language Assessment Quarterly","FirstCategoryId":"98","ListUrlMain":"https://doi.org/10.1080/15434303.2021.1895162","RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"LANGUAGE & LINGUISTICS","Score":null,"Total":0}
引用次数: 6

Abstract

ABSTRACT A long-standing debate in the testing of listening concerns the authenticity of the listening input. On the one hand, listening texts produced by item writers often lack spoken language characteristics. On the other hand, real-life recordings are often too context-specific to stand alone, or not suitable for item generation. In this study, we explored the effectiveness of an existing item-writing training course to produce authentic-sounding listening texts within the constraints of test specifications. Twenty-five trainees took an online item-writing course including training on creating authentic-sounding listening texts. Prior to and after the course, they developed a listening task. The resulting listening texts were judged on authenticity by three professional item reviewers and analysed linguistically by the researchers. Additionally, we interviewed the trainees following each item writing event and analysed their online discussions from during the course. Statistical comparison of the pre-and post-course authenticity scores revealed a positive effect of the training on item-writers’ ability to produce authentic-sounding listening texts, while the linguistic analysis demonstrated that the texts produced after the training contained more instances of spoken language. The interviews and discussions revealed that item writers’ awareness of spoken language features and their text production techniques influenced their ability to develop authentic-sounding texts.
听力评估中的文本真实性:项目作者是否可以接受训练来制作听起来真实的文本?
听力测试中一个长期存在的争论是关于听力输入的真实性。一方面,项目编写者创作的听力文本往往缺乏口语特征。另一方面,现实生活中的记录往往过于特定于上下文而不能单独存在,或者不适合用于项目生成。在这项研究中,我们探讨了现有的项目写作培训课程在测试规范的约束下产生听起来真实的听力文本的有效性。25名受训者参加了在线项目写作课程,其中包括编写听起来真实的听力文本的培训。在课程之前和之后,他们制定了一个听力任务。最终的听力文本由三位专业的项目评审员对真实性进行评判,并由研究人员进行语言分析。此外,我们在每个项目写作活动之后采访了学员,并分析了他们在课程期间的在线讨论。课程前和课程后真实性分数的统计比较表明,训练对项目作者创作听起来真实的听力文本的能力有积极影响,而语言学分析表明,训练后产生的文本包含更多的口语实例。访谈和讨论表明,项目编写者对口语特征的认识和他们的文本制作技术影响了他们开发听起来真实的文本的能力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
6.40
自引率
3.40%
发文量
22
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信