Reinforcement learning algorithm for reusable resource allocation with unknown rental time distribution

Abstract

We explore a scenario where a platform must decide on the price and type of reusable resources for sequentially arriving customers. The product is rented for a random period, during which the platform also extracts rewards based on a prearranged agreement. The expected reward varies during the usage time, and the platform aims to maximize revenue over a finite horizon. Two primary challenges are encountered: the stochastic usage time introduces uncertainty, affecting product availability, and the platform lacks initial knowledge about reward and usage time distributions. In contrast to conventional online learning, where usage time distributions are parametric, our problem allows for unknown distribution types. To overcome these challenges, we formulate the problem as a Markov decision process and model the usage time distribution using a hazard rate. We first introduce a greedy policy in the full-information setting with a provable 1/2-approximation ratio. We then develop a reinforcement learning algorithm to implement this policy when the parameters are unknown, allowing for non-parametric distributions and time-varying rewards. We further prove that the algorithm achieves sublinear regret against the greedy policy. Numerical experiments on synthetic data as well as a real dataset from TikTok demonstrate the effectiveness of our method.