Users’ search performance prediction in cross-device search

Abstract

Users’ search performance indicates the effectiveness and success with which users’ information needs are met, which is calculated based on the relevance judgment by users themselves. This study proposed to explore the prediction of users’ search performance in the context of cross-device search. A user experiment was performed to collect users’ relevance judgments and search behaviors in cross-device search. Based on users’ relevance judgments, users’ search performance was evaluated by calculating the percentage of valid clicks, effective search time, nDCG@n, and satisfaction. A simple linear regression model was adopted to train the prediction model. The final results showed that a combination of users’ search performance in pre-switch sessions and their search behavior in post-switch sessions can attain the best prediction accuracy. Important features to predict users’ search performance in cross-device search shed light on improving search systems to aid users in completing the task efficiently.