{"title":"Examining the Scoring of Content Integration in a Listening-Speaking Test: A G-Theory Analysis","authors":"Rongchan Lin","doi":"10.1080/15434303.2023.2242334","DOIUrl":"https://doi.org/10.1080/15434303.2023.2242334","url":null,"abstract":"ABSTRACT Communication in the real world often entails the interpretation, evaluation, and integration of content from different sources. However, it appears that the ability to integrate content into discourse has not been explicitly scored for in existing studies. This study operationalizes content integration in the analytic scoring of a listening-speaking test in Chinese. International students who were non-native speakers of Chinese took the test that comprised two retell tasks and an oral presentation linked by an academic scenario. They were scored for content integration, organization, delivery, and language control for all three tasks. Multivariate generalizability theory (G-theory) was used to investigate the functioning of content integration in the analytic rubric of the retell tasks and oral presentation respectively. Overall, this study aimed to illuminate issues on dependability and construct validity for the two task types, focusing on content integration particularly. The findings suggested that content integration functioned differently to some extent when compared with the other dimensions studied.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44781591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nathalie L. Marinho, S. Witmer, Nicole Jess, S. Roschmann
{"title":"A Meta-Analysis of Accommodation Effects for English Learners: Considering Possible Moderators","authors":"Nathalie L. Marinho, S. Witmer, Nicole Jess, S. Roschmann","doi":"10.1080/15434303.2023.2237499","DOIUrl":"https://doi.org/10.1080/15434303.2023.2237499","url":null,"abstract":"ABSTRACT The use of accommodations is often recommended to remove barriers to academic testing among English Learners (ELs). However, it is unclear whether accommodations are particularly effective at improving ELs’ test scores. A growing foundation of empirical work has explored this topic. We conducted a meta-analysis that examined several possible moderators of accommodation effectiveness for improving EL test performance. Results showed substantial variability among estimates, an overall small positive significant effect on test scores and a statistically significant positive effect for indirect linguistic supports and combined accommodations compared to native language accommodations.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42273933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Comparability Between Graded Readers and the Common Test in Japan in Terms of Text Difficulty Perceived by Learners","authors":"Yuya Arai","doi":"10.1080/15434303.2023.2237497","DOIUrl":"https://doi.org/10.1080/15434303.2023.2237497","url":null,"abstract":"ABSTRACT Proponents of extensive reading argue that test-oriented language teaching and learning could discourage extensive reading practice. However, this has not been examined empirically. Given the possibility that the consistency of reading texts used in extensive reading and entrance examinations is one source of washback, the present study employed a two-facet Rasch measurement to examine the comparability between graded readers and the Common Test in Japan in terms of text difficulty perceived by Japanese high school students. It was found that perceived difficulty of the Common Test texts was not statistically different from that of some graded reader texts, providing positive evidence for the consistency between the Common Test and graded reader texts. Implications for future research on the relationship between entrance examinations and extensive reading are discussed in detail based on study findings and limitations.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48061279","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Canadian English Language Proficiency Index Program (CELPIP) Test","authors":"Melissa McLeod, Liying Cheng","doi":"10.1080/15434303.2023.2237487","DOIUrl":"https://doi.org/10.1080/15434303.2023.2237487","url":null,"abstract":"ABSTRACT The Canadian English Language Proficiency Index Program (CELPIP) Test was designed for immigration and citizenship in Canada. CELPIP is a computer-based English-language proficiency test which covers all four skills. This test review provides a description of the test and its construct, tasks, and delivery. Then, it appraises CELPIP for reliability, fairness, and validity before identifying future directions for research.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46730674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using Causal Explanation Speaking Tasks to Assess Young EFL learners’ Speaking Ability: The Effects of Age, Cognitive, and L2 Linguistic Development","authors":"Wenjun Ding, Guoxing Yu","doi":"10.1080/15434303.2023.2206811","DOIUrl":"https://doi.org/10.1080/15434303.2023.2206811","url":null,"abstract":"ABSTRACT This paper examined to what extent causal explanation speaking tasks (CESTs) are cognitively appropriate for assessing young language learners’ (YLLs) L2 speaking. Ninety-six YLLs (48 from Grade 4 and 6 each) in China performed two CESTs in both L1 (Chinese) and L2 (English). They also completed receptive and productive L2 vocabulary size tests. We examined how their CEST performance scores, choice of causal antecedents, and speech utterances were related with language modes of the tasks (L1 vs. L2), grade levels, and L2 vocabulary sizes. L2 CEST performance scores were found to have significant positive correlations with L2 productive vocabulary size. CESTs were found to be generally cognitively appropriate for YLLs because their high scores in L1 performance indicated that performing CESTs is within their L1 capacity. By examining causal connectives used by YLLs, we found that learners from both age groups had cognitive ability sufficient to verbalise causality. Yet YLLs’ cognitive ability to interpret and verbalise mental states is still developing and reasoning between causal antecedents that have competing causal relationship with the final state can be cognitively challenging. We discussed the findings with reference to the design of cognitively appropriate CESTs that can assess both language and thinking skills. 本文研究了因果解释口语任务从认知层面来说是否适宜测评儿童语言学习者的二语口语能力。九十六名中国的儿童语言学习者(四年级与六年级各四十八名)分别用第一语言(汉语)和第二语言(英语)完成了两个因果解释口语任务。他们还完成了接受性和产出性二语词汇测试。我们分析了他们的因果解释口语任务的表现分数、对前因的选择和语言产出这三者与完成任务所使用的语言 (一语和二语)、所在年级和二语词汇量之间的关系。我们发现因果解释口语任务的二语表现分数与二语产出性词汇量之间呈显著正相关。他们用第一语言完成因果解释口语任务所获得的高分数说明完成这项任务在他们的一语能力范围之内,因此我们认为因果解释口语任务对于儿童语言学习者来说总体上是认知适宜的。通过分析儿童语言学习者使用因果连词的情况,我们还发现这两个年龄组的学习者拥有充分的认知能力来口头表达因果关系。不过儿童语言学习者理解和口头表达心理状态的能力尚处于发展阶段,他们对具有相互竞争关系的前因进行逻辑推导时会存在认知上的挑战。我们对研究结果的讨论主要涉及如何设计认知适宜的并能够同时测评语言和思维技能的因果解释口语任务。","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42183412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning-Oriented Language Assessment: Putting Theory into Practice","authors":"Fatimah M. M. Aseeri","doi":"10.1080/15434303.2023.2181174","DOIUrl":"https://doi.org/10.1080/15434303.2023.2181174","url":null,"abstract":"","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46188165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Teaching and Testing Second Language Pragmatics and Interaction: A Practical Guide","authors":"Dunlai Lin, Senlin Liu","doi":"10.1080/15434303.2023.2181173","DOIUrl":"https://doi.org/10.1080/15434303.2023.2181173","url":null,"abstract":"Pragmatics — how social context affects the way we do things with language — plays a critical role in managing social interaction and interpersonal relationships. Despite decades of research on second language (L2) pragmatics, practical resources for teaching and assessing L2 pragmatics have continued to remain relatively scarce. Teaching and testing second language pragmatics and interaction: A practical guide by Roever (2022) is the first comprehensive hands-on guide written for language practitioners who wish to interweave aspects of pragmatics into their classroom teaching, curriculum design","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48546277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Aligning Language Frameworks: An Example with the CLB and CEFR","authors":"B. North, E. Piccardo","doi":"10.1080/15434303.2023.2184266","DOIUrl":"https://doi.org/10.1080/15434303.2023.2184266","url":null,"abstract":"ABSTRACT This paper presents a methodology for directly aligning ‘can do’ frameworks to each other. The methodology, inspired by the manual for relating examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (CEFR) (Council of Europe, 2009) and Kane’s (2004, 2013) interpretative argument, takes account of both the horizontal dimension (content analysis) and the vertical dimension (benchmarking with Multifaceted Rasch Modelling – MFRM). The paper exemplifies the application of the methodology by introducing the research conducted to align the Canadian Language Benchmarks (CLB)/ Niveaux de compétence linguistique canadiens (NCLC) to the CEFR, presenting the resulting alignment, and discussing the rationale for the choices made.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2023-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49338693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Promoting Learning Potential among Students of L2 Chinese through Dynamic Assessment","authors":"Lin Jia, Jian-Yong Cai, Jianqi Wang","doi":"10.1080/15434303.2022.2156870","DOIUrl":"https://doi.org/10.1080/15434303.2022.2156870","url":null,"abstract":"ABSTRACT In Dynamic Assessment (DA), the observation that individuals respond differently to support, or mediation, is important for diagnoses of development. The concept of learning potential refers to openness to mediation, i.e., the extent of change to performance when mediation is available, which may suggest learners will need less overall instruction to develop. The current study investigates this prediction and the premise that aligning DA mediation with learner needs promotes development. Mediation, as a systematic interactional process, is more likely to appropriately identify learner needs that are sensitive to future development. Thirty-four secondary school learners of L2 Chinese participating in a two-month study abroad in China completed narration tasks following an instructional intervention, one group receiving DA mediation and the other explicit feedback. The bα (把)-construction, recognized as particularly challenging for learners of Chinese, was the focal language feature. Analysis revealed significant differences, with the group receiving DA mediation showing greater improvement with mediation and more accuracy during independent functioning.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2022-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47276982","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Advancing L2 Dynamic Assessment: Innovations in Chinese Contexts","authors":"M. E. Poehner, J. Lantolf","doi":"10.1080/15434303.2022.2158465","DOIUrl":"https://doi.org/10.1080/15434303.2022.2158465","url":null,"abstract":"ABSTRACT This introduction to the special issue, L2 Dynamic Assessment Research in China, examines the theoretical foundations of Dynamic Assessment (DA) in the writings of L. S. Vygotsky, with particular attention to the concepts of praxis, mediation, and zone of proximal development, while also recognizing contributions from notable DA researchers such as Israeli psychologist and educator Reuven Feuerstein. Trends in the general and L2 DA research literatures are considered, including: flexible, open-ended mediation (interactionist DA) and standardized, scripted mediation (interventionist DA); formats of embedding mediation in assessment procedures (so-called ‘sandwich’ and ‘cake’ formats of DA); DA administered in classroom and group settings; DA in formal testing contexts, including computerized DA procedures; mediated scores and learning potential scores; and uses of learner profiles resulting from DA to inform instructional enrichment programs. Each article in the special issue reports original research conducted by scholars in China. How the individual studies take up and extend trends in L2 DA research is explained. Given that learning activities and the thinking associated with them are deeply saturated in specific cultural practices and norms, it is argued that extension of DA principles to new cultural contexts, such as those reported in the special issue, is informative not only for assessment researchers in China but for the international community of assessment scholars, and it is essential for the continued development of DA frameworks.","PeriodicalId":46873,"journal":{"name":"Language Assessment Quarterly","volume":null,"pages":null},"PeriodicalIF":2.9,"publicationDate":"2022-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46414186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}