The situatedness of instructional quality-How situated are college students' ratings?

Abstract

Background: Student ratings are commonly used to evaluate classroom processes. Research suggests that these ratings are not solely based on objective situational characteristics but also reflect rater characteristics. Although research has been conducted in K-12 settings, little is known about how person-specific and situation-specific factors affect student ratings in higher education.

Aims: This study investigates to what extent students' ratings on instructional quality (INQ) vary in response to various data collection across teaching situations. In this way, we also investigate the role of motivational beliefs in these ratings.

Sample: The study uses data from 1,745 undergraduates (26.5% self-identified as male students) from a university in California, U.S.A.

Methods: We employed single trait-multistate models (STMS) to decompose INQ variance (classroom management, student support, cognitive activation). We estimate the consistency and situation specificity of ratings across courses, time, and course type. Also, we estimated correlations between motivational beliefs and the consistent component across ratings, i.e., the person effect.

Results: High consistency in student ratings of INQ was found within the same course over time. This consistency was lower when INQ was rated across different courses. The consistency was higher in courses that students perceived as difficult than to courses that were perceived as important. Students who reported higher expectancies for success and interest value in these courses also rated INQ more positive across teaching situations.

Conclusions: Our findings suggest that student ratings of INQ are situated and shaped by course characteristics but also rater characteristics. Motivational beliefs contribute uniquely to students' INQ ratings beyond situational characteristics.