An effective implicit relevance feedback technique using affective, physiological and behavioural features

The effectiveness of various behavioural signals for implicit relevance feedback models has been exhaustively studied. Despite the advantages of such techniques for a real time information retrieval system, most of the behavioural signals are noisy and therefore not reliable enough to be employed. Among many, a combination of dwell time and task information has been shown to be effective for relevance judgement prediction. However, the task information might not be available to the system at all times. Thus, there is a need for other sources of information which can be used as a substitute for task information. Recently, affective and physiological signals have shown promise as a potential source of information for relevance judgement prediction. However, their accuracy is not high enough to be applicable on their own. This paper investigates whether affective and physiological signals can be used as a complementary source of information for behavioural signals (i.e. dwell time) to create a reliable signal for relevance judgement prediction. Using a video retrieval system as a use case, we study and compare the effectiveness of the affective and physiological signals on their own, as well as in combination with behavioural signals for the relevance judgment prediction task across four different search intentions: seeking information, re-finding a particular information object, and two different entertainment intentions (i.e. entertainment by adjusting arousal level, and entertainment by adjusting mood). Our experimental results show that the effectiveness of studied signals varies across different search intentions, and when affective and physiological signals are combined with dwell time, a significant improvement can be achieved. Overall, these findings will help to implement better search engines in the future.