A Validated Computer Terminology Test for Predicting Non-native English-speaking CS1 Students’ Academic Performance

Abstract

This research examines the validity and efficacy of an English computer terminology multiple-choice test. The test was developed to investigate the relationship between non-native English-speaking students’ knowledge of computer terminology and their academic performance in CS1 courses. In this research, the Rasch model was used to find empirical evidence of test validity. Six aspects of construct validity were evaluated: content, substantive, structural, generalisability, external and consequential aspects. Based on analyses of 150 students’ responses, the test shows an acceptable level of validity in all six aspects of construct validity. Moreover, the efficacy of using this test to predict the students’ academic performance in CS1 courses was investigated. Statistical analysis of the data shows that the computer terminology test can predict approximately 50% of final exam performance in CS1 courses after controlling the impact of English proficiency. The aim of this research (along with other ongoing research) is to aid in the identification of non-native English-speaking students who require more specific language support. Additionally, the research will guide the development of resources and tools that can help these students.