Customer Choice Models vs. Machine Learning: Finding Optimal Product Displays on Alibaba

We compare the performance of two approaches for finding the optimal set of products to display to customers landing on Alibaba's two online marketplaces, Tmall and Taobao. Both approaches were placed online simultaneously and tested on real customers for one week. The first approach we test is Alibaba's current practice. This procedure embeds thousands of product and customer features within a sophisticated machine-learning algorithm that is used to estimate the purchase probabilities of each product for the customer at hand. Our second approach uses a featurized multinomial logit (MNL) model to predict purchase probabilities for each arriving customer. In this way, we use less sophisticated machinery to estimate purchase probabilities, but we employ a model that was built to capture customer purchasing behavior and, more specifically, substitution patterns. Our experiments show that despite the lower prediction power of our MNL-based approach, it generates significantly higher revenue per visit compared with the current machine-learning algorithm with the same set of features.