Personalized multimedia content retrieval through relevance feedback techniques for enhanced user experience

Abstract

Emerging multimedia interactive services inherently call for user-centered design approaches, where the involved high degree of interactivity requires the implementation of efficient and effective information retrieval approaches. In this paper, a multimodal content retrieval framework is introduced that employs personalization along with relevance feedback techniques in order to enhance provided QoE, by retrieving and offering multimedia content tailored to individual users' characteristics and/or preferences. The developed Relevance Feedback mechanism engages the user into assessing the relevance of the initially retrieved results list of the original query, and through one or more iterations to present him with the most relevant result list based on his feedback. Our proposed framework implements a similarity learning scheme to improve multimedia content retrieval, towards increasing user experience. A model for implicit relevance feedback is formulated and a confidence level parameter is introduced to classify the results, based on the Jaccard similarities of the results that did not receive explicit feedback with those that did. This relevance feedback mechanism acts complementary to the personalized search by reranking the initial retrieved set in iterative rounds. The performance and effectiveness of the proposed framework was evaluated and demonstrated through an extensive experimental study, utilizing an interactive multimodal multimedia web-based system, with media files consisting of a set of 3D movies containing audio-visual content with high and low level semantic annotations.