Does Knowledge of Clinical Case Outcome Influence Supervisor Evaluation of Resident Clinical Reasoning?

Abstract

Purpose: This study examines whether outcome bias affects the assessment of general practice (GP) residents and explores supervisor feedback characteristics.

Method: In a within-subjects experiment conducted in June 2023, Erasmus Medical Center GP supervisors reviewed 6 clinical vignettes with ambiguous diagnoses assessing residents' diagnostic decisions. Each vignette had 2 versions, differing only in the final sentence indicating favorable or adverse clinical outcome. Supervisors were randomly assigned to review half the vignettes with favorable clinical outcomes and half with adverse clinical outcomes. Supervisors provided scores (range of 1-10, with 10 indicating exceptional achievement and 1-5 indicating insufficient performance) and feedback, analyzed for valence, content specificity, process versus outcome focus, and politeness strategies.

Results: Sixty-two supervisors participated in the study. Vignettes ending in adverse clinical outcomes received lower scores versus those ending in favorable clinical outcomes (mean [SE] scores, 5.25 [0.12] vs 6.26 [0.16]; P < .001) and prompted more negative feedback (mean [SE] negative idea units, 2.35 [0.11] vs 1.80 [0.09]; P < .001). Negative feedback exhibited greater specificity than positive feedback (mean [SE] proportion of specific idea units, 0.88 [0.02] vs 0.44 [0.03]; P < .001), regardless of clinical outcome. Most feedback addressed process-related aspects (grand mean proportion of process-related idea units, 0.97; 95% CI, 0.95-0.98). Polite language was more prevalent in negative vs positive feedback (mean [SE] proportion of feedback with politeness strategies, 0.50 [0.04] vs 0.15 [0.02]; P < .001), regardless of clinical outcome.

Conclusions: The study identified outcome bias in the evaluation of GP residents, with adverse clinical outcomes leading to lower scores and more negative, specific, process-focused feedback. Although such feedback can enhance learning, it may also hinder learning by triggering negative emotions. The findings emphasize the educational value of diagnostic errors but stress the need for objective assessment strategies to optimize learning opportunities.