T. Joachims, Maria Dimakopoulou, Adith Swaminathan, Yves Raimond, Olivier Koch, Flavian Vasile
{"title":"REVEAL 2019: closing the loop with the real world: reinforcement and robust estimators for recommendation","authors":"T. Joachims, Maria Dimakopoulou, Adith Swaminathan, Yves Raimond, Olivier Koch, Flavian Vasile","doi":"10.1145/3298689.3346975","DOIUrl":"https://doi.org/10.1145/3298689.3346975","url":null,"abstract":"The REVEAL workshop1 focuses on framing the recommendation problem as a one of making personalized interventions. Moreover, these interventions sometimes depend on each other, where a stream of interactions occurs between the user and the system, and where each decision to recommend something will have an impact on future steps and long-term rewards. This framing creates a number of challenges we will discuss at the workshop. How can recommender systems be evaluated offline in such a context? How can we learn recommendation policies that are aware of these delayed consequences and outcomes?","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133735309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multi-armed recommender system bandit ensembles","authors":"Rocío Cañamares, Marcos Redondo, P. Castells","doi":"10.1145/3298689.3346984","DOIUrl":"https://doi.org/10.1145/3298689.3346984","url":null,"abstract":"It has long been found that well-configured recommender system ensembles can achieve better effectiveness than the combined systems separately. Sophisticated approaches have been developed to automatically optimize the ensembles' configuration to maximize their performance gains. However most work in this area has targeted simplified scenarios where algorithms are tested and compared on a single non-interactive run. In this paper we consider a more realistic perspective bearing in mind the cyclic nature of the recommendation task, where a large part of the system's input is collected from the reaction of users to the recommendations they are delivered. The cyclic process provides the opportunity for ensembles to observe and learn about the effectiveness of the combined algorithms, and improve the ensemble configuration progressively. In this paper we explore the adaptation of a multi-armed bandit approach to achieve this, by representing the combined systems as arms, and the ensemble as a bandit that at each step selects an arm to produce the next round of recommendations. We report experiments showing the effectiveness of this approach compared to ensembles that lack the iterative perspective. Along the way, we find illustrative pitfall examples that can result from common, single-shot offline evaluation setups.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"201 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134217944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
O. Alkan, Massimiliano Mattetti, E. Daly, A. Botea, Inge Vejsbjerg
{"title":"IRF: interactive recommendation through dialogue","authors":"O. Alkan, Massimiliano Mattetti, E. Daly, A. Botea, Inge Vejsbjerg","doi":"10.1145/3298689.3346966","DOIUrl":"https://doi.org/10.1145/3298689.3346966","url":null,"abstract":"Recent research focuses beyond recommendation accuracy, towards human factors that influence the acceptance of recommendations, such as user satisfaction, trust, transparency and sense of control. We present a generic interactive recommender framework that can add interaction functionalities to non-interactive recommender systems. We take advantage of dialogue systems to interact with the user and we design a middleware layer to provide the interaction functions, such as providing explanations for the recommendations, managing users' preferences learnt from dialogue, preference elicitation and refining recommendations based on learnt preferences.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133481001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, A. Nath, Shawn Andrews, A. Kumthekar, M. Sathiamoorthy, Xinyang Yi, Ed H. Chi
{"title":"Recommending what video to watch next: a multitask ranking system","authors":"Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, A. Nath, Shawn Andrews, A. Kumthekar, M. Sathiamoorthy, Xinyang Yi, Ed H. Chi","doi":"10.1145/3298689.3346997","DOIUrl":"https://doi.org/10.1145/3298689.3346997","url":null,"abstract":"In this paper, we introduce a large scale multi-objective ranking system for recommending what video to watch next on an industrial video sharing platform. The system faces many real-world challenges, including the presence of multiple competing ranking objectives, as well as implicit selection biases in user feedback. To tackle these challenges, we explored a variety of soft-parameter sharing techniques such as Multi-gate Mixture-of-Experts so as to efficiently optimize for multiple ranking objectives. Additionally, we mitigated the selection biases by adopting a Wide & Deep framework. We demonstrated that our proposed techniques can lead to substantial improvements on recommendation quality on one of the world's largest video sharing platforms.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123502145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"PAL","authors":"Huifeng Guo, Jinkai Yu, Q. Liu, Ruiming Tang, Yuzhou Zhang","doi":"10.1145/3298689.3347033","DOIUrl":"https://doi.org/10.1145/3298689.3347033","url":null,"abstract":"Predicting Click-Through Rate (CTR) accurately is crucial in recommender systems. In general, a CTR model is trained based on user feedback which is collected from traffic logs. However, position-bias exists in user feedback because a user clicks on an item may not only because she favors it but also because it is in a good position. One way is to model position as a feature in the training data, which is widely used in industrial applications due to its simplicity. Specifically, a default position value has to be used to predict CTR in online inference since the actual position information is not available at that time. However, using different default position values may result in completely different recommendation results. As a result, this approach leads to sub-optimal online performance. To address this problem, in this paper, we propose a Position-bias Aware Learning framework (PAL) for CTR prediction in a live recommender system. It is able to model the position-bias in offline training and conduct online inference without position information. Extensive online experiments are conducted to demonstrate that PAL outperforms the baselines by 3% - 35% in terms of CTR and CVR (ConVersion Rate) in a three-week AB test.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116388780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Francisco Gutiérrez, Sven Charleer, Robin De Croon, N. Htun, Gerd Goetschalckx, K. Verbert
{"title":"Explaining and exploring job recommendations: a user-driven approach for interacting with knowledge-based job recommender systems","authors":"Francisco Gutiérrez, Sven Charleer, Robin De Croon, N. Htun, Gerd Goetschalckx, K. Verbert","doi":"10.1145/3298689.3347001","DOIUrl":"https://doi.org/10.1145/3298689.3347001","url":null,"abstract":"The dynamics of the labor market and the tasks with which jobs are being composed are continuously evolving. Job mobility is not evident, and providing effective recommendations in this context has also been found to be particularly challenging. In this paper, we present Labor Market Explorer, an interactive dashboard that enables job seekers to explore the labor market in a personalized way based on their skills and competences. Through a user-centered design process involving job seekers and job mediators, we developed this dashboard to enable job seekers to explore job recommendations and their required competencies, as well as how these competencies map to their profile. Evaluation results indicate the dashboard empowers job seekers to explore, understand, and find relevant vacancies, mostly independent of their background and age.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124612544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leveraging post-click feedback for content recommendations","authors":"Hongyi Wen, Longqi Yang, D. Estrin","doi":"10.1145/3298689.3347037","DOIUrl":"https://doi.org/10.1145/3298689.3347037","url":null,"abstract":"Implicit feedback (e.g., clicks) is widely used in content recommendations. However, clicks only reflect user preferences according to their first impressions. They do not capture the extent to which users continue to engage with the content. Our analysis shows that more than half of the clicks on music and short videos are followed by skips from two real-world datasets. In this paper, we leverage post-click feedback, e.g. skips and completions, to improve the training and evaluation of content recommenders. Specifically, we experiment with existing collaborative filtering algorithms and find that they perform poorly against post-click-aware ranking metrics. Based on these insights, we develop a generic probabilistic framework to fuse click and post-click signals. We show how our framework can be applied to improve pointwise and pairwise recommendation models. Our approach is shown to outperform existing methods by 18.3% and 2.5% respectively in terms of Area Under the Curve (AUC) on the short-video and music dataset. We discuss the effectiveness of our approach across content domains and trade-offs in weighting various user feedback signals.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121047930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Recommendation in home improvement industry, challenges and opportunities","authors":"Khalifeh Al Jadda","doi":"10.1145/3298689.3346960","DOIUrl":"https://doi.org/10.1145/3298689.3346960","url":null,"abstract":"Retail industry has been disrupted by the e-commerce revolution more than any other industry. Some giant retailers went out of business or filed for bankruptcy as a result of that like Sears and Toys R Us. However, some verticals in the retail industry are still robust and not been disrupted due to the lack of e-commerce solutions that convinced customers to turn their back to the existing physical stores in favor of the online experience. Home improvement is the best example of such vertical where e-commerce has not \"yet\" disrupted the domain and caused problems to the leading companies which still rely heavily on physical stores. That being said, home improvement retailers recognized the risk of not investing in building a robust online business that support their physical stores in a seamless experience so most of the leading retailers in this hundred-billion-dollar industry started building their in-house solutions for all the challenging problems to give their shoppers a seamless experience when they shop online. Recommender systems playing crucial role in this industry like any other online retailers. Therefore, it is very important to invest in building personalized, scalable, and reliable recommender system that proactively help shoppers discover products that engage them and match their intent and interest while on the website then reengage them with products and content that align with their interest after they leave the website via email or social media. As a Sr. Manager of Core Recommendations team at The Home Depot which is the largest home improvement retailer in the world, I deal with the challenges of building such recommender system utilizing the cutting-edge technologies in AI, machine learning, and data science. In this talk I would like to discuss and highlight the following challenges in the recommendations for home improvement: (1) Project-based recommendations: One of the unique aspects on home improvement retail is project-based shopping. Most of the visitors of home improvement retails are classified as \"Do It Yourself\" where those customers who are non-home improvement professionals, but they are interested in building or fixing something in their home themselves. For those customers they prefer to go to the physical store most of the time, so they can talk to a store associate about their project and get the associate help in getting the needed tools and materials for their project. It is very challenging to build similar experience online so I will talk about what we have done at Home Depot to build a project-based recommendation utilizing multi-modal learning to achieve that goal. (2) Item Related Groups (IRG): One of the most important recommendations on the home improvement portals is the Item Related Groups (IRG) which includes accessories (water filter is an accessory for a fridge), collections (faucet has shower head, towel bar, and towel ring which match the style as collection), and Parts (handler of a drawer).","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"627 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115673560","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Domain adaptation in display advertising: an application for partner cold-start","authors":"Karan Aggarwal, Pranjul Yadav, S. Keerthi","doi":"10.1145/3298689.3347004","DOIUrl":"https://doi.org/10.1145/3298689.3347004","url":null,"abstract":"Digital advertisements connects partners (sellers) to potentially interested online users. Within the digital advertisement domain, there are multiple platforms, e.g., user re-targeting and prospecting. Partners usually start with re-targeting campaigns and later employ prospecting campaigns to reach out to untapped customer base. There are two major challenges involved with prospecting. The first challenge is successful on-boarding of a new partner on the prospecting platform, referred to as partner cold-start problem. The second challenge revolves around the ability to leverage large amounts of re-targeting data for partner cold-start problem. In this work, we study domain adaptation for the partner cold-start problem. To this end, we propose two domain adaptation techniques, SDA-DANN and SDA-Ranking. SDA-DANN and SDA-Ranking extend domain adaptation techniques for partner cold-start by incorporating sub-domain similarities (product category level information). Through rigorous experiments, we demonstrate that our method SDA-DANN outperforms baseline domain adaptation techniques on real-world dataset, obtained from a major online advertiser. Furthermore, we show that our proposed technique SDA-Ranking outperforms baseline methods for low CTR partners.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126288572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Meng Wu, Ying Zhu, Qilian Yu, Bhargav Rajendra, Yunqi Zhao, Navid Aghdaie, Kazi A. Zaman
{"title":"A recommender system for heterogeneous and time sensitive environment","authors":"Meng Wu, Ying Zhu, Qilian Yu, Bhargav Rajendra, Yunqi Zhao, Navid Aghdaie, Kazi A. Zaman","doi":"10.1145/3298689.3347039","DOIUrl":"https://doi.org/10.1145/3298689.3347039","url":null,"abstract":"The digital game industry has recently adopted recommender systems to deliver the most relevant content and suggest the most suitable activities to players. Because of diverse game designs and dynamic experiences, recommender systems typically operate in highly heterogeneous and time-sensitive environments. In this paper, we describe a recommender system at a digital game company which aims to provide recommendations for a large variety of use-cases while being easy to integrate and operate. The system leverages a unified data platform, standardized context and tracking data pipelines, robust naive linear contextual multi-armed bandit algorithms, and experimentation platform for extensibility as well as flexibility. Several games and applications have successfully launched with the recommender system and have achieved significant improvements.","PeriodicalId":215384,"journal":{"name":"Proceedings of the 13th ACM Conference on Recommender Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124253072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}