Correlating perception-oriented aspects in user-centric recommender system evaluation

Abstract

Research on recommender systems evaluation generally measures the quality of the algorithm, or system, offline, i.e. based on some information retrieval metric, e.g. precision or recall. The metrics do however not always reflect the users' perceptions of the recommendations. Perception-related values are instead often measured through user studies, however the bulk of the work on recommender systems is evaluated through offline analysis. In the work presented in this paper we choose to neglect the quality of the recommender system and instead focus on the similarity of aspects related to users' perception of recommender systems. Based on a user study (N = 132) we show the correlation of concepts such as usefulness, ratings, obviousness, and serendipity from the users' perspectives.