Multi-agent reinforcement learning with causal communication for ride-sourcing pricing in mixed autonomy mobility

Abstract

The burgeoning self-driving technology has provided a solid impetus for the ride-sourcing market and new demand and supply management challenges. Under the context of a long-haul mixed operation of autonomous vehicles and human-driven vehicles, this paper focuses on profit-maximizing pricing for both demand and supply sides, in which the prices are differentiated by service type, time, and location. Diverging from most studies limited to centralized control for small-scale problems, we align with distributed and scalable requirements in practice and tackle the coordination challenge from a causal communication perspective. Based on the spatial supply–demand interdependencies inherent in the ride-sourcing market, operation areas are modeled as collaborative intelligent agents. The pricing problem is formulated as a decentralized partially observable Markov game augmented with neighborhood communication. Then a multi-agent reinforcement learning with causal communication method is developed to jointly optimize pricing policy and communication mechanism through end-to-end learning. The bidirectional communication mechanism is ensured to be effective and succinct by maximizing the causal effect of the communication message. Leveraging theoretical analysis, the proposed method is proven to cope with partial observability and non-stationary environments through collaborative communication. Besides, an agent-based simulator for mixed autonomy mobility is established on a real-world large-scale network, emulating the causal communication process among decentralized areas, as well as the heterogeneity, elasticity, and uncertainty of ride-sourcing demand and supply. Two representative scenarios are designed to demonstrate the dynamic evolutions of mixed autonomy mobility: (a) smaller-sized autonomous vehicles and conservative passenger acceptance (conservative stage), and (b) larger-sized autonomous vehicles and liberal passenger acceptance (liberal stage). The results highlight that incorporating the causal communication mechanism can speed up the learning process and guide informed pricing decisions. Furthermore, the proposed method gains managerial insights into proactively regulating pricing schemes for a smooth transformation into fully autonomous ride-sourcing services.