User response to two algorithms as a test of collaborative filtering

Abstract

The purpose of this experiment was to determine whether recommendations based on collaborative filtering (CF) are perceived as superior to recommendations based on user population averages. The test vehicle was a movie recommender. 29 subjects were divided into 2 groups, each group using one of these systems. The recommneder systems suggested movies which subjects later viewed. Each subject filled out pre and post-questionnaires about their experience. Subjects using the CF algorithm rated more movies. Subjects placed slightly more confidence in the recommendations of the population averages algorithm. Both algorithms were over-confident compared to subjects ratings. Subjects found both recommender systems to be an effective source of finding entertainment. User responses did not reveal a noticeable difference between the two algorithms.