Language Testing最新文献_第4页

Practical considerations when building concordances between English tests 在英语测试之间建立一致性时的实际考虑

1区文学

Language Testing Pub Date : 2023-09-23 DOI: 10.1177/02655322231195027

Ramsey L. Cardwell, Steven W. Nydick, J.R. Lockwood, Alina A. von Davier

{"title":"Practical considerations when building concordances between English tests","authors":"Ramsey L. Cardwell, Steven W. Nydick, J.R. Lockwood, Alina A. von Davier","doi":"10.1177/02655322231195027","DOIUrl":"https://doi.org/10.1177/02655322231195027","url":null,"abstract":"Applicants must often demonstrate adequate English proficiency when applying to postsecondary institutions by taking an English language proficiency test, such as the TOEFL iBT, IELTS Academic, or Duolingo English Test (DET). Concordance tables aim to provide equivalent scores across multiple assessments, helping admissions officers to make fair decisions regardless of the test that an applicant took. We present our approaches to addressing practical (i.e., data collection and analysis) challenges in the context of building concordance tables between overall scores from the DET and those from the TOEFL iBT and IELTS Academic tests. We summarize a novel method for combining self-reported and official scores to meet recommended minimum sample sizes for concordance studies. We also evaluate sensitivity of estimated concordances to choices about how to (a) weight the observed data to the target population; (b) define outliers; (c) select appropriate pairs of test scores for repeat test takers; and (d) compute equating functions between pairs of scores. We find that estimated concordance functions are largely robust to different combinations of these choices in the regions of the proficiency distribution most relevant to admissions decisions. We discuss implications of our results for both test users and language testers.","PeriodicalId":17928,"journal":{"name":"Language Testing","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135967326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Book review: C. A. Chapelle and E. Voss (Eds.), Validity Argument in Language Testing: Case Studies of Validation Research 书评：C.A.Chapelle和E.Voss（编辑），《语言测试中的有效性争论：验证研究的案例研究》

IF 4.1 1区文学

Language Testing Pub Date : 2023-08-17 DOI: 10.1177/02655322231193705

Yasuyo Sawaki

引用次数: 0

Language testers and their place in the policy web 语言测试人员及其在政策网络中的地位

IF 4.1 1区文学

Language Testing Pub Date : 2023-08-17 DOI: 10.1177/02655322231191133

Laura Schildt, B. Deygers, A. Weideman

引用次数: 0

Assessing the content quality of essays in content and language integrated learning: Exploring the construct from subject specialists’ perspectives 评估内容和语言综合学习中论文的内容质量:从学科专家的角度探索结构

IF 4.1 1区文学

Language Testing Pub Date : 2023-08-08 DOI: 10.1177/02655322231190058

Takanori Sato

{"title":"Assessing the content quality of essays in content and language integrated learning: Exploring the construct from subject specialists’ perspectives","authors":"Takanori Sato","doi":"10.1177/02655322231190058","DOIUrl":"https://doi.org/10.1177/02655322231190058","url":null,"abstract":"Assessing the content of learners’ compositions is a common practice in second language (L2) writing assessment. However, the construct definition of content in L2 writing assessment potentially underrepresents the target competence in content and language integrated learning (CLIL), which aims to foster not only L2 proficiency but also critical thinking skills and subject knowledge. This study aims to conceptualize the construct of content in CLIL by exploring subject specialists’ perspectives on essays’ content quality in a CLIL context. Eleven researchers of English as a lingua franca (ELF) rated the content quality of research-based argumentative essays on ELF submitted in a CLIL course and produced think-aloud protocols. This study explored some essay features that have not been considered relevant in language assessment but are essential in the CLIL context, including the accuracy of the content, presence and quality of research, and presence of elements required in academic essays. Furthermore, the findings of this study confirmed that the components of content often addressed in language assessment (e.g., elaboration and logicality) are pertinent to writing assessment in CLIL. The manner in which subject specialists construe the content quality of essays on their specialized discipline can deepen the current understanding of content in CLIL.","PeriodicalId":17928,"journal":{"name":"Language Testing","volume":" ","pages":""},"PeriodicalIF":4.1,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48912792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Test review: High-stakes English language proficiency tests—Enquiry, resit, and retake policies 考试回顾：高风险的英语语言能力测试——查询、补考和重考政策

IF 4.1 1区文学

Language Testing Pub Date : 2023-07-25 DOI: 10.1177/02655322231186706

William S. Pearson

{"title":"Test review: High-stakes English language proficiency tests—Enquiry, resit, and retake policies","authors":"William S. Pearson","doi":"10.1177/02655322231186706","DOIUrl":"https://doi.org/10.1177/02655322231186706","url":null,"abstract":"Many candidates undertaking high-stakes English language proficiency tests for academic enrolment do not achieve the results they need for reasons including linguistic unreadiness, test unpreparedness, illness, an unfavourable configuration of tasks, or administrative and marking errors. Owing to the importance of meeting goals or out of a belief that original test performance was satisfactory, some individuals query their results, while others go on to retake the test, perhaps on multiple occasions. This article critically reviews the policies of eight well-known, on-demand gatekeeping English language tests, describing the systems adopted by language assessment organisations to regulate results enquiries, candidates resitting (components of) a test where performance fell short of requirements, and repeat test-taking. It was found that all providers institute clear mechanisms through which candidates can query their results, with notable variations exhibited in procedures, costs, restrictions, outcomes, and how policies are communicated to test-takers. Test resit options are scarce, while organisations enact few restrictions on test retakes in the form of mandatory waiting times and cautionary advice. The implications for language assessment organisations are discussed.","PeriodicalId":17928,"journal":{"name":"Language Testing","volume":" ","pages":""},"PeriodicalIF":4.1,"publicationDate":"2023-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41921024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Book review: K. Sadeghi (Ed.), Technology-Assisted Language Assessment in Diverse Contexts: Lessons from the Transition to Online Testing During Covid-19 书评：K.Sadeghi（编辑），《不同背景下的技术辅助语言评估：新冠肺炎期间向在线测试过渡的经验教训》

IF 4.1 1区文学

Language Testing Pub Date : 2023-07-20 DOI: 10.1177/02655322231186707

Tomohito Hiromori, H. Mohebbi

引用次数: 0

Assessing speaking through multimodal oral presentations: The case of construct underrepresentation in EAP contexts 通过多模式口头陈述评估口语：EAP语境中结构代表性不足的案例

IF 4.1 1区文学

Language Testing Pub Date : 2023-07-07 DOI: 10.1177/02655322231183077

Louise Palmour

引用次数: 0

Development of the American Sign Language Fingerspelling and Numbers Comprehension Test (ASL FaN-CT) 美国手语手指拼写与数字理解测验(ASL FaN-CT)

IF 4.1 1区文学

Language Testing Pub Date : 2023-07-03 DOI: 10.1177/02655322231179494

C. Occhino, Ryan Lidster, Leah Geer, Jason D. Listman, P. Hauser

{"title":"Development of the American Sign Language Fingerspelling and Numbers Comprehension Test (ASL FaN-CT)","authors":"C. Occhino, Ryan Lidster, Leah Geer, Jason D. Listman, P. Hauser","doi":"10.1177/02655322231179494","DOIUrl":"https://doi.org/10.1177/02655322231179494","url":null,"abstract":"We describe the development and initial validation of the “ASL Fingerspelling and Number Comprehension Test” (ASL FaN-CT), a test of recognition proficiency for fingerspelled words in American Sign Language (ASL). Despite the relative frequency of fingerspelling in ASL discourse, learners commonly struggle to produce and perceive fingerspelling more than they do other facets of ASL. However, assessments of fingerspelling knowledge are highly underrepresented in the testing literature for signed languages. After first describing the construct, we describe test development, piloting, revisions, and evaluate the strength of the test’s validity argument vis-à-vis its intended interpretation and use as a screening instrument for current and future employees. The results of a pilot on 79 ASL learners provide strong evidence that the revised test is performing as intended and can be used to make accurate decisions about ASL learners’ proficiency in fingerspelling recognition. We conclude by describing the item properties observed in our current test, and our plans for continued validation and analysis with respect to a battery of tests of ASL proficiency currently in development.","PeriodicalId":17928,"journal":{"name":"Language Testing","volume":" ","pages":""},"PeriodicalIF":4.1,"publicationDate":"2023-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49114525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Fairness of using different English accents: The effect of shared L1s in listening tasks of the Duolingo English test 使用不同英语口音的公平性：Duolingo英语测试听力任务中共享L1的效果

IF 4.1 1区文学

Language Testing Pub Date : 2023-07-03 DOI: 10.1177/02655322231179134

Okim Kang, Xun Yan, M. Kostromitina, Ron I. Thomson, T. Isaacs

{"title":"Fairness of using different English accents: The effect of shared L1s in listening tasks of the Duolingo English test","authors":"Okim Kang, Xun Yan, M. Kostromitina, Ron I. Thomson, T. Isaacs","doi":"10.1177/02655322231179134","DOIUrl":"https://doi.org/10.1177/02655322231179134","url":null,"abstract":"This study aimed to answer an ongoing validity question related to the use of nonstandard English accents in international tests of English proficiency and associated issues of test fairness. More specifically, we examined (1) the extent to which different or shared English accents had an impact on listeners’ performances on the Duolingo listening tests and (2) the extent to which different English accents affected listeners’ performances on two different task types. Speakers from four interlanguage English accent varieties (Chinese, Spanish, Indian English [Hindi], and Korean) produced speech samples for “yes/no” vocabulary and dictation Duolingo listening tasks. Listeners who spoke with these same four English accents were then recruited to take the Duolingo listening test items. Results indicated that there is a shared first language (L1) benefit effect overall, with comparable test scores between shared-L1 and inner-circle L1 accents, and no significant differences in listeners’ listening performance scores across highly intelligible accent varieties. No task type effect was found. The findings provide guidance to better understand fairness, equality, and practicality of designing and administering high-stakes English tests targeting a diversity of accents.","PeriodicalId":17928,"journal":{"name":"Language Testing","volume":" ","pages":""},"PeriodicalIF":4.1,"publicationDate":"2023-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45816789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

English learners who are blind or visually impaired: A participatory design approach to enhancing fairness and validity for language testing accommodations 盲人或视障英语学习者:提高语言测试公平性和有效性的参与式设计方法

IF 4.1 1区文学

Language Testing Pub Date : 2023-06-27 DOI: 10.1177/02655322231159143

Danielle Guzman-Orth, Jonathan Steinberg, Traci Albee

{"title":"English learners who are blind or visually impaired: A participatory design approach to enhancing fairness and validity for language testing accommodations","authors":"Danielle Guzman-Orth, Jonathan Steinberg, Traci Albee","doi":"10.1177/02655322231159143","DOIUrl":"https://doi.org/10.1177/02655322231159143","url":null,"abstract":"Standardizing accessible test design and development to meet students’ individual access needs is a complex task. The following study provides one approach to accessible test design and development using participatory design methods with school community members. Participatory research provides opportunities to empower collaborators by co-creating knowledge that is useful for assessment development. In this study, teachers of students who are visually impaired, students who are blind or are visually impaired, English language teachers, and test administrators provided feedback at critical stages of the development process to explore the construct validity of English language proficiency (ELP) assessments. Students who are blind or visually impared need to be able to show what they know and can do without impact from construct-irrelevant variance like language acquisition or disability characteristics. Building on our iterative accessible test design, development, and delivery practices, and as part of a large project on English-learner proficiency test accessibility and usability, we collected rich observation and interview data from 17 students who were blind or visually impaired and were enrolled in grades kindergarten through Grade 12. We examined the ratings and item metadata, including assistive technology preferences and interactions, while we used grounded theory approaches to examine qualitative thematic findings. Implications for research and practice are discussed.","PeriodicalId":17928,"journal":{"name":"Language Testing","volume":"40 1","pages":"933 - 959"},"PeriodicalIF":4.1,"publicationDate":"2023-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43968301","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1