Automate Page Layout Optimization: An Offline Deep Q-Learning Approach

Abstract

The modern e-commerce web pages have brought better customer experience and more profitable services by whole page optimization at different granularity, e.g., page layout optimization, item ranking optimization, etc. Generating the proper page layout per customer’s request is one of the vital tasks during the web page rendering process, which can directly impact customers’ shopping experience and their decision-making. In this paper, we formulate the request-rendering interactions as a Markov decision process (MDP) and solve it by deep reinforcement learning (RL). Specifically, we present the design and implementation of applying offline Deep Q-Learning (DQN) to the contextual page layout optimization problem. Through the offline evaluation method, we demonstrate the effectiveness of the proposed framework, i.e., the RL agent has the potential to perform better than the baseline ranker by learning from the offline data set, e.g., the RL agent can improve the average cumulative rewards up to 36.69% comparing to the baseline ranker.