An eye-tracking and neuroimaging study of negative wording effects on cognitive load in a metacognitive awareness tool

We compared the extraneous cognitive load imposed by two wording conditions: negatively worded and non-negated items in an L2 questionnaire. The questionnaire is the metacognitive awareness listening questionnaire (MALQ), a widely used instrument for assessing metacognitive awareness strategies. Respondents' (N = 109) eye movements measured by an eye-tracker and brain activation levels measured by functional near-infrared spectroscopy (fNIRS) were obtained to examine their extraneous cognitive load in responding to MALQ. Using MANOVA, we identified distinct gaze behavior associated with negatively worded items, indicating that the presence of such items may increase extraneous cognitive load. In addition, English-as-a-second-language participants generally exhibited higher extraneous cognitive load than their L1 counterparts. The results of the neuroimaging investigation further indicated that negatively worded items require more visual attention and cognitive effort, while positively worded items may engage higher-order reflective processing in the brain, highlighting a complex interaction between attention and extraneous cognitive load. Additionally, linear mixed effects models showed that although the models under the two-wording conditions explained a significant amount of variation in respondents’ MALQ scores, they had relatively lower explanatory power (R²) though better global fit compared to the models based on the five constructs that MALQ measures (planning-evaluation, directed attention, person knowledge, mental translation, and problem-solving). We suggest that a balanced approach that considers both the target constructs and the negeative wording effect might be the most effective strategy in questionnaire design and validation. Further implications of these findings are discussed.