Investigating test content structure using multidimensional scaling

Abstract

Although multidimensional scaling (MDS) has extensively been used in social and physical sciences to visualize and explore the latent dimensionality of observed data, few studies in language assessment have used MDS for the purpose of test construction, including construct and content validation. In this study, we use MDS to investigate and compare the similarity of writing prompts in the IELTS and TOEFL iBT tests. Random prompts from both tests were presented as item pairs and raters were asked to rate the similarity of the pairs in terms of content and cognitive complexity. The results showed the writing prompts in the TOEFL iBT test represented two major dimensions while those in the IELTS test demonstrated three domains.