Investigating LID in four test conditions - Do instructions, test formats and item positioning matter?

Abstract

Recent research has found the Updated Vocabulary Levels Test (UVLT) to have Local Item Dependence (LID), a violation to the central assumption of all Rasch and Item Response Theory models. LID in the UVLT is hypothesized to be caused by a feature of matching tasks: once an option is selected for one target word, it will not be selected for another. It is also hypothesized that if this feature is removed, LID will be reduced. The present study investigated the effects of LID in four test conditions. The first employed the 3:6 matching format of the UVLT with no instruction concerning option recycling. The second used the same format but with instructions encouraging option recycling. The third utilized a multiple-choice format, with items belonging to the same UVLT cluster using identical sets of 6 options and placed adjacently. The fourth also used a multiple-choice, 6-option format, but items sharing identical options were far apart, making them less “local”. Data from 231 Vietnamese EFL learners were analyzed using Rasch unidimensional modelling and Rasch Testlet Modelling (RTM). Person estimates from the unidimensional models and the general dimensions from the RTMs were compared and correlated. Substantial LID was present in Conditions 1–3. Significant distortions of person estimates were found in all test conditions. However, the findings showed that LID had a negligible impact on person ordering in all test conditions.