Proceedings of the 31st ACM Conference on Hypertext and Social Media最新文献

筛选

英文中文

Off-line vs. On-line Evaluation of Recommender Systems in Small E-commerce 小型电子商务中推荐系统的离线与在线评价

Proceedings of the 31st ACM Conference on Hypertext and Social Media Pub Date : 2018-09-10 DOI: 10.1145/3372923.3404781

Ladislav Peška, P. Vojtás

{"title":"Off-line vs. On-line Evaluation of Recommender Systems in Small E-commerce","authors":"Ladislav Peška, P. Vojtás","doi":"10.1145/3372923.3404781","DOIUrl":"https://doi.org/10.1145/3372923.3404781","url":null,"abstract":"In this paper, we present our work towards comparing on-line and off-line evaluation metrics in the context of small e-commerce recommender systems. Recommending on small e-commerce enterprises is rather challenging due to the lower volume of interactions and low user loyalty, rarely extending beyond a single session. On the other hand, we usually have to deal with lower volumes of objects, which are easier to discover by users through various browsing/searching GUIs. The main goal of this paper is to determine applicability of off-line evaluation metrics in learning true usability of recommender systems (evaluated on-line in A/B testing). In total 800 variants of recommenders were evaluated off-line w.r.t. 18 metrics covering rating-based, ranking-based, novelty and diversity evaluation. The off-line results were afterwards compared with on-line evaluation of 12 selected recommender variants and based on the results, we tried to learn and utilize an off-line to on-line results prediction model. Off-line results shown a great variance in performance w.r.t. different metrics with the Pareto front covering 64% of the approaches. Furthermore, we observed that on-line results are considerably affected by the seniority of users. On-line metrics correlates positively with ranking-based metrics (AUC, MRR, nDCG) for novice users, while too high values of novelty had a negative impact on the on-line results for them.","PeriodicalId":389616,"journal":{"name":"Proceedings of the 31st ACM Conference on Hypertext and Social Media","volume":"151 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123400174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 19

首页上一页