Decision support through deep reinforcement learning for maximizing a courier's monetary gain in a meal delivery environment

Abstract

Meal delivery is a fast-growing industry supported by couriers participating in the gig economy. This paper takes a single courier's perspective and provides decision support for an individual courier who works at will in repositioning between jobs and order-taking to optimize her profit during a work period. A hybrid discrete-time, discrete-event simulation environment was developed based on data from a real-world meal delivery environment to replicate daily operations. The single courier's repositioning and order-taking decision problem is formulated as a Markov decision process. Two classes of deep reinforcement learning (DRL) methodologies, value-based and policy-gradient algorithms, were implemented to determine the courier's best decisions to take as the courier's work shift progresses. In numerical experiments, the best optimal policy resulting from the DRL algorithms is shown to outperform all considered static policies in all demand environments. Insights from studying the decisions suggested by the best of the DRL methods were employed to create a promising static policy by generating decision trees for relocation and order-taking. The results indicate that as couriers find more intelligent strategies for maximizing their rewards, the meal delivery platform will have even greater need to incentivize couriers to fulfill less attractive orders, especially in surge periods. Finally, the impact of a multi-courier DRL environment, where multiple couriers have the advantage of the DRL strategy, was studied. For this purpose, a multi-agent DRL was implemented and numerical experiments were conducted to investigate the tradeoffs between individual courier gains and system-level performance. Findings from this multi-agent extension show the negative impacts of selfish behavior on not only the system, but the couriers themselves.