Studies in Language Assessment最新文献

Investigating cognitive processes in different item formats in reading tests through eye-tracking and verbal protocols 通过眼动追踪和语言协议研究阅读测试中不同项目格式的认知过程

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/uixk2220

Hatice Akgün, A. Ünaldı

引用次数: 0

College English Test–Spoken English Test (CET-SET) 大学英语考试(CET-SET)

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/oofm7473

Lin Zhang

引用次数: 10

Exploring shared and individual assessment of paired oral interactions 探讨配对口头互动的共同和个人评估

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/homq5772

Pakize Uludag, Kim McDonough, P. Trofimovich

{"title":"Exploring shared and individual assessment of paired oral interactions","authors":"Pakize Uludag, Kim McDonough, P. Trofimovich","doi":"10.58379/homq5772","DOIUrl":"https://doi.org/10.58379/homq5772","url":null,"abstract":"Studies concerning the assessment of second language (L2) paired oral interaction to date have investigated interactional patterns that emerge from paired oral tests and identified the factors that create variability in an individual’s test scores. However, less work has addressed concerns about whether scores should be shared or individual when assessing L2 speakers’ oral interactions. Therefore, the present study compared shared and individual assessment of L2 English paired oral task performances. Paired oral interaction episodes were sampled from a larger corpus of university-level L2 speakers engaged in paired speaking tasks and were assessed by 60 raters who were randomly assigned to rate Speaker A, Speaker B, or both speakers. To avoid any possible rating effects due to rating stimuli, half the raters evaluated audio recordings while the other half assessed video recordings. The raters used an analytic rubric with four domains: discourse management, collaborative communication, content development, and language accuracy and complexity. Comparison of raters’ scores revealed that individual discourse management ratings were significantly higher than shared ratings for both members of the pair regardless of the rating modality (audio vs. video). Implications for assessing pair interactions are discussed.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81053547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Gebril, A. (Ed.) Learning-Oriented Language Assessment: Putting Theory into Practice. (编)《以学习为导向的语言评估:理论与实践》。

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/pbmv5990

A. Berger

引用次数: 0

The meaningfulness of two curriculum-based national tests of English as a foreign language 两个以课程为基础的国家外语英语考试的意义

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/dqvs8821

R. Hildén, Johanna Pulkkinen, Juhani Rautopuro

{"title":"The meaningfulness of two curriculum-based national tests of English as a foreign language","authors":"R. Hildén, Johanna Pulkkinen, Juhani Rautopuro","doi":"10.58379/dqvs8821","DOIUrl":"https://doi.org/10.58379/dqvs8821","url":null,"abstract":"This paper addresses the aspect of the meaningfulness of a national assessment in English as a foreign language, applying the fairness framework proposed by Kunnan (2018). We compared students’ performance on two receptive language skills of listening and reading on two subsequent national evaluations in Finland, taken by students at the end of compulsory basic education and at the end of general upper secondary education, respectively. The research questions focus on (1) the relationship between students’ performance on the two tests and their gender, language of schooling and parents’ educational level, and (2) the relationship between the students’ receptive language proficiency at the end of basic education and general upper secondary education. The data were analysed using linear regression and quantile regression analyses. The effect of background variables on the proficiency was stronger for low-performing students than for high-performing students. Moreover, students’ proficiency on the receptive skills at the end of basic education well predicted that at the end of general upper secondary education across several points in the score distribution. The findings also indicated persistent challenges with respect to educational equality and equity that requires ongoing attention by policy-makers and test designers.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76424925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Implications of training university teachers in developing local writing rating scales 培训大学教师编制地方写作评定量表的意义

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/xvdf9070

O. Kvasova, Lyudmyla Hnapovska, V. Kalinichenko, Luliia Budas

{"title":"Implications of training university teachers in developing local writing rating scales","authors":"O. Kvasova, Lyudmyla Hnapovska, V. Kalinichenko, Luliia Budas","doi":"10.58379/xvdf9070","DOIUrl":"https://doi.org/10.58379/xvdf9070","url":null,"abstract":"Language assessment literacy is currently in search of new, modern conceptualisations in which contextual factors have a growing significance and impact (Tsagari, 2020). This article presents an initiative to promote writing assessment literacy in a culture-specific educational context. Assessment of writing belongs to the under-researched areas in Ukrainian higher education, wherein teachers have to act as raters and as rating scale developers without being properly trained in language assessment. The gaps in writing assessment literacy prompted research into the strengths and weaknesses of using a local rating scale developed by university teachers. It was conducted within an Erasmus + Staff mobility project in 2016-2019 and followed up by dissemination events held in several universities in Ukraine. The сurrent study aims to explore the impact of training in writing assessment on the processes and outcomes of university teachers’ development and use of analytic rating scales. The paper analyses how three teams of teachers from different universities coped with the task, and whether the training they underwent enabled them to design well-performing rating scales. The nine participants in the study developed three local context-specific analytic rating scales following the intuitive method of scale design, detailed in the guidelines prepared by the trainer. Given the same context (ESP) and the same CEFR level (B1 ->B2), we managed to compare the three local rating scales. The study testifies to a positive impact of the training on teachers’ literacy in writing assessment.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88518060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Language assessment literacy of language teachers in the context of adult education in Spain 西班牙成人教育背景下语言教师的语言评价素养

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/rrdv2344

Julia Consuelo Zabala Delgado, Cristina Rodriguez

{"title":"Language assessment literacy of language teachers in the context of adult education in Spain","authors":"Julia Consuelo Zabala Delgado, Cristina Rodriguez","doi":"10.58379/rrdv2344","DOIUrl":"https://doi.org/10.58379/rrdv2344","url":null,"abstract":"Recent years have seen a growth in the use of language tests as measures of accountability within the education sector. In many countries, governmental institutions have promoted the involvement of teachers in language testing, providing training to boost the language assessment literacy (LAL) of teachers. This study aims to analyse the results of a large-scale effort to increase teachers’ LAL within the context of public language education for adults in Spain by shedding light on the scale and nature of teacher LAL, the impact of training as perceived by teachers, and their self-perceived further needs. Results show that, similarly to other countries in which teacher LAL has been studied, training in assessment is strongly influenced by contextual factors. Moreover, teachers perceive that this training has an impact not only on assessment-related tasks, but on their general teaching practice. Lastly, the findings reveal a significant correlation between the contents of assessment training courses and the teachers’ perception of further training needs. This could indicate that the more teachers learn about specific areas of language assessment, the more training in assessment they feel is needed, suggesting a gap in teachers’ awareness of their own LAL that materialises once training is provided.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81054209","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Contextual variables in written assessment feedback in a university-level Spanish program 大学水平西班牙语课程书面评估反馈中的上下文变量

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/aanz

Kathryn Hill, A. Ducasse

{"title":"Contextual variables in written assessment feedback in a university-level Spanish program","authors":"Kathryn Hill, A. Ducasse","doi":"10.58379/aanz","DOIUrl":"https://doi.org/10.58379/aanz","url":null,"abstract":"The ‘situated’ nature of assessment may help to explain why feedback interventions are successful in one setting but not in another. This study reanalyses data from an earlier study (Ducasse & Hill, 2019) using a coherent theory of context, Bronfenbrenner’s ecological framework (1979), to investigate contextual influences on teacher feedback practices and learner responses respectively. Participants comprised 15 beginner, intermediate and advanced level students in an Australian university Spanish language program. Data comprised summative feedback on writing tasks and audio-recordings and transcripts of teacher and student think-aloud protocols, and discussions. Data were analysed in relation to the five systems of Bronfenbrenner’s framework (microsystem, mesosystem, exosystem, macrosystem and chronosystem). The study found teacher practices appeared to be influenced by perceptions of the learners in addition to task and performance characteristics (microsystem), by institutional policies and practices (exosystem) and by external professional accreditation standards (macrosystem). Contextual factors found to influence learner responses included perceptions of the teacher, the timing and valence of feedback (microsystem), course level and maturity as a learner (chronosystem) as well as by other subjects the learners were enrolled in (mesosystem). The study demonstrates the utility of Bronfenbrenner’s framework for systematic reflection on contextual influences in language assessment research.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72491304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Language assessment literacy: Understanding the construct from Norwegian EFL teachers’ perspective 语言评价素养:从挪威英语教师的视角看语言评价素养的建构

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/unud5510

Henrik Bøhn, Dina Tsagari

{"title":"Language assessment literacy: Understanding the construct from Norwegian EFL teachers’ perspective","authors":"Henrik Bøhn, Dina Tsagari","doi":"10.58379/unud5510","DOIUrl":"https://doi.org/10.58379/unud5510","url":null,"abstract":"The question of what language assessment literacy (LAL) entails continues to preoccupy the research community. In the current study empirical evidence from Norway was gathered in order to inform our understanding of the construct. The focus of the investigation was twofold: First, it explored how Norwegian EFL teachers understand LAL. Secondly, it examined to what extent the same teachers found the dimensions in Taylor’s (2013) LAL model relevant in the Norwegian context. To that end semi-structured interviews were carried out with 10 teachers at the lower and upper secondary levels in Norway. The results showed that the teachers attributed a lot of weight to disciplinary competence and formative assessment, and they agreed with Taylor on several of her hypothesized dimensions, such as the importance given to language pedagogy. However, they also disagreed with some of the other dimensions, such as test-specific skills (e.g., how to design and validate multiple-choice tests), which they believed to be less relevant in the Norwegian context. The findings point to the contextualized nature of teacher LAL and provide important knowledge on how teacher LAL can be conceptualized and how Taylor’s model can be developed and refined.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77405474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Is the devil you know better? Testwiseness and eliciting evidence of interactional competence in familiar versus unfamiliar triadic speaking tasks 你更了解魔鬼吗?熟悉与不熟悉三合一口语任务中互动能力的测试性和引出性证据

Studies in Language Assessment Pub Date : 2022-01-01 DOI: 10.58379/ttfe6660

Nichola Glasson

{"title":"Is the devil you know better? Testwiseness and eliciting evidence of interactional competence in familiar versus unfamiliar triadic speaking tasks","authors":"Nichola Glasson","doi":"10.58379/ttfe6660","DOIUrl":"https://doi.org/10.58379/ttfe6660","url":null,"abstract":"Research has proven the impact testwiseness and test preparation can have on speaking test discourse (Lam, 2015; Luk, 2010; Hüttner, 2014). The question is perhaps not whether assessment leads to contrived interaction but how interactions may differ between a more familiar task (face-to-face Cambridge B2 First Speaking) and an entirely new online task. This conversation analytic case study of a group of three in two different tasks (one face-to-face, one online) sought to explore how testwiseness is implicated in candidate talk and to explicate differences in what a more familiar and a completely new task elicited. The analytic focus was on the conversational object “I agree” but includes reference to non-verbal behaviours, turn-taking practices and the relationship between agreement and topic shift. The findings indicate a contrast in terms of closure and progressivity – that is, an observable difference in the closure of topics and the forward movement of conversation – across the two tasks. The data illustrates how the same candidates performed very differently in two tasks in the space of 40 minutes. In so doing, it raises questions about broadening the potential assessments have to target interactional competence.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75409036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0