Response-shift effect for self-assessment: Magnitude and accuracy within an interprofessional education course

Abstract

Objective

Response-shift has described a shift in respondents' self-assessment following an impactful experience—where a respondent may initially score themselves higher than their true ability on Pre-assessment, though shifted/lowered their initial ability-score (on Post-assessment) after they learned further. While the psychological literature had described this ‘response-shift’ effect decades ago, our objective at this time was to evaluate the magnitude that response-shift was occurring in our interprofessional education (IPE) context and using our own self-assessment instrument.

Methods

Using a standard response-shift study design, first-year health-professions students within a foundational IPE course completed beginning-of-course (Pre) and end-of-course (Post) self-assessments based on that IPE course's student-learning-objectives; additionally at course-end, students completed a retrospective-beginning-of-course (RetroPre) self-assessment (to reflect and score themselves retrospectively for course's beginning). Paired t-tests compared each student's Post minus Pre (Conventional) versus Post minus RetroPre (Adjusted-effect); effect-sizes used Cohen's-d.

Results

Among 267 students, conventional change had a 0.5 effect-size (medium), while adjusted-effect change was 1.5 (very-large). Notably, some students’ conventional change was negative, though none of those same students reported negative-change in their adjusted-effect.

Conclusion

We found a substantial response-shift in our educational context, and evidence for improved accuracy. Moreover, RetroPre/Post appeared more efficient for students (as they could complete both in one sitting at course-end, instead of twice with conventional Pre/Post), and staff (no one needed to match each student's Pre and Post assessments).