Uncovering Systematic Bias in Ratings across Categories: a Bayesian Approach

Abstract

Recommender systems are routinely equipped with standardized taxonomy that associates each item with one or more categories or genres. Although such information does not directly imply the quality of an item, the distribution of ratings vary greatly across categories, e.g. animation movies may generally receive higher ratings than action movies. While it is a natural outcome given the diversity and heterogeneity of both users and items, it makes directly aggregated ratings, which are commonly used to guide users' choice by reflecting the overall quality of an item, incomparable across categories and hence prone to fairness and diversity issues. This paper aims to uncover and calibrate systematic category-wise biases for discrete-valued ratings. We propose a novel Bayesian multiplicative probit model that treats the inflation or deflation of mean rating for a combination of categories as multiplicatively contributed from category-specific parameters. The posterior distribution of those parameters, as inferred from data, can capture the bias for all possible combinations of categories, thus enabling statistically efficient estimation and principled rating calibration.