Rating Level and Accuracy as a Function of Rater Personality

Abstract

We investigated the predictability of rating level and two measures of rating accuracy from rater Agreeableness (A) and Conscientiousness (C) scores of the Five Factor Model. One hundred and twenty-six students made peer ratings after participating in several group exercises under conditions designed to emulate the modal peer rating system in which raters had low accountability for their ratings. Scores were correlated with average rating level (r=.18, p<.05) and both measures of rating accuracy (p<.05) and C scores were correlated with rating level (−.20, p<.05) and both measures of rating accuracy (p<.05). As suggested by Bernardin, Villanova, and Cooke (Journal of Applied Psychology, 85, 232–236, 2000), raters who were both more agreeable and less conscientious made the most lenient and least accurate ratings. Contrary to Yun, Donahue, Dudley, and McFarland (International Journal of Selection and Assessment, 13, 97–107, 2005), more agreeable raters also tended to rate the least effective performers more leniently than did other raters.