How Up-to-date should it be? the Value of Instant Profiling and Adaptation in Information Filtering

In profile-based or content-based adaptive systems, one of the open research questions is how frequently the user's profile and the list of recommended items should be updated. Different systems tend to choose one of the two extremes. Some systems do it once per session (thus called between-session update strategy), whereas some others update whenever there is feedback (called instant update strategy). This paper presents our attempt to assess the value of keeping the list of recommended items up-to-date in the context of task-based information exploration. We conducted controlled studies involving human users performing realistic tasks using two systems that have the same adaptive filtering engine but with the above two different update strategies. Our results show that the between-session strategy helped to find better quality information, and received better subjects' responses about its usefulness and usability. However, it prolonged the selection of useful passages, whereas the instant update strategy helped subjects to obtain almost all of their selected passages (>98%) within the first 5 minutes. Based on the results, we hypothesize that the best strategy for updating might be a hybrid between the two update strategies, where both adaptability and stability can be achieved.