Ongoing Faculty Evaluations: Developmental Gain or Just More Pain?

Purpose

We continue to increase the amount of evaluations to improve the outcomes of our residency programs. Although ongoing faculty evaluations clearly are an important part of faculty development, their value in terms of improving the program needs to be evaluated. The questions asked were as follows: (1) Do faculty evaluations continue to improve the faculty over the course of successive evaluation periods? (2) Are there groups of faculty who would benefit the most from faculty evaluation feedback? (3) Are there any specific objective categories within the evaluation that carry more value and may help to shorten this form?

Methods

Forty-two faculty members were evaluated by 40 surgical residents with an assessment form developed by surgical residents that assessed faculty members by 10 different criteria. The initial set of data was collected, and attending surgeons were given an intervention in the form of a letter detailing how they had been assessed in each of the 10 categories. The attending surgeons were evaluated again 6 months later and were given an intervention in the form of verbal feedback regarding their evaluations. The attending faculty members were then assessed 1 year after that. One way analyses of variance and Fisher Protected Least Significant Difference (PLSD) were used to analyze the resulting data to determine if there were significant differences in the faculty evaluations. A part–whole correlation was performed that correlated the 10 evaluation criteria against the mean score on each evaluation, and partial η-squared analysis was used to determine which criterion had the largest effect on the overall means.

Results

The means for the 42 faculty members as a whole continued to improve from the first to the final evaluation period, with 30 faculty members increasing their mean score (18 significantly) and 12 decreasing their mean score (4 significantly). Seven of the 10 evaluation criteria’s means improved sequentially by feedback session. These were (1) Didactic Teaching, (2) Teaching Rounds, (3) Attendance at Didactic Activities, (4) Allows [Resident] Autonomy to Make Independent Decisions, (5) Provides Feedback, (6) Stimulates Critical Thinking with Use of Literature, and (7) Encourages and Maintains an Atmosphere of Professional Mutual Respect for All Members of Health Care Team (Role Model).

The faculty group with the lowest evaluations improved significantly more than those of both the middle and the role model group, with the middle and the role model groups improving, but not significantly differently from each other.

Of the criteria that improved, only Provides Feedback improved significantly from the first to second and the second to third evaluation periods. The three criteria with the highest correlation coefficients were Role Model (0.76), Provides Feedback (0.75), and Stimulates Critical Thinking (0.74). The results from the partial η-squared test showed that the criterion with the largest effect size was Provides Feedback (0.28). These analyses indicate that the criterion Provides Feedback was both highly correlated with the average score on a faculty member’s evaluation and was more responsible than any other criteria for the overall improvement in the mean evaluation score of the faculty members.

Conclusions

(1) Ongoing faculty evaluations indeed are a powerful tool to improve the faculty as a whole. (2) The faculty members with the lowest evaluations showed the largest amount of improvement. (3) Providing feedback to the residents seems to be the most valued factor by the residents for faculty evaluations and perhaps could become the basis of the evaluation for the most accomplished faculty.