Effect of Anchoring Vignettes on Reliability and Validity as a Measure of Motivation and Emotion in Mathematics

In mathematics, the importance of emotion and motivation has long been appreciated. Meanwhile, the measurement of these constructs needs to be carefully developed. Indeed, many previous studies employed questionnaires and self-rating to investigate emotion and motivation in mathematics. However, researchers stressed that self-rating and questionnaires are vulnerable to different biases, such as cultural bias, social desirability bias, and individual reporting behavior, which could influence the reliability and validity of the responses. The present study employed an anchoring vignette approach and self-rating to measure emotion and motivation to examine its effect on reliability and validity. The study sample consisted of 308 Grade 8 students from urban, suburban, and rural schools in Mongolia. The students were administered two vignette sets and their respective self-rating items in 45 minutes. The study found that the anchoring vignette approach increased two types of reliability coefficients, including composite reliability and Cronbach's α, and two types of validity, such as convergent and discriminant validity. In sum, the results of the present study concluded that the anchoring vignette approach has the potential to improve the reliability and validity of tools to measure emotion and motivation in mathematics.