Comparison Between Holistic and Analytic Rubrics of a Paired Oral Test

Abstract

The current study aimed to reveal similarities and differences between a holistic and an analytic rubric used in assessing speaking performance in a paired oral test. To this end, speaking performances of 110 Japanese university students produced in paired oral interaction were evaluated by raters, holistically and analytically. The comparisons made between the two rubrics using many-facet Rasch measurement showed that both worked effectively, with the analytic rubric working slightly better in terms of a better global fit, a better test-taker and task separation, higher test-taker and task reliability, smaller standard errors, and a smaller percentage of test takers with overfit. Correlation and regression analysis indicated a strong relationship between the two (r = .84) and the Interactive communication and Fluency analytic criteria substantially explained holistic scores (adjusted R2 = .71). Results suggest that teachers can obtain similar results with either rubric type and, if they select an analytic one, a priority would be to include Interactive communication and Fluency criteria.