Validity in High- and Low-Stakes Tests: A Comparison of Academic Vocabulary and some Lexical Features in CLIL and non-CLIL Students’ Written Texts

Abstract

In second language (L2) learning research, learners’ proficiency levels and progress are often investigated. Sometimes high-stakes tests, which are part of the school curriculum, are used for this purpose, but more often tests designed for the purpose of the specific research study are utilized. How do we know that tests of the latter kind actually show what learners know and can do, when they do not have any impact on school grades? In other words, how can we be sure that our informants do as well in low-stakes tests specifically designed for research purposes as they would in high-stakes tests that result in final grades, and thus have an impact on the individual’s future? The answer is, of course, that we can never know for sure. One way of finding out, though, is to compare results from high- and low-stakes tests. In this study, we examine whether students display similar levels of performance when writing in high- and low-stakes contexts, with regard to the use of English academic vocabulary and some other linguistic features, more precisely text length, word length and variation of vocabulary. Thereby, we indirectly explore whether students have put a similar amount of effort into high- and low-stakes writing assignments. We investigate this by analyzing and comparing texts written under high- and low-stakes conditions. The purpose of the study is, firstly, to validate results obtained in the low-stakes writing assignments used in the large-scale longitudinal research project Content and Language Integration in Swedish Schools, CLISS, focusing in particular on results regarding productive academic vocabulary and the linguistic features mentioned above. Secondly, we hope that this study will shed new light on validity in relation to writing assignments in high- and low-stakes contexts in a more general sense, for instance with regard to the role of effort and motivation.