Recommending items to users: an explore/exploit perspective

Abstract

Content recommendation is the science of algorithmically recommending engaging content to users interacting with the web in various contexts. Information on WWW is consumed in various forms that may range from web search results displayed in response to an explicit query on one hand, to unsolicited advertisements that attract user attention on the other extreme. Between these two extremes, there is a continuum of contexts where users visit websites in the browse mode without a specific task in mind but expecting a certain mix of content. Examples include visits to sites like news, sports, videos, blog pages and so on. Traditionally, content in such contexts have been served through editorial oversight to ensure high quality and adhere to typical mix associated with a website. Such a process is not scalable in large scale modern web applications, it is also difficult to efficiently incorporate various measurable metrics like clicks and downstream engagement in recommending content. Further, deeply personalized content recommendation that is desirable in various applications is simply not possible through such a manual process. This clearly underscores the importance of using algorithmic methods to optimize content that we discuss in this review article. We begin by discussing why content recommendation is a key technology to enable engaging user interactions on WWW in many contexts. We then provide a mathematical