Multi-Agent Reinforcement Learning for truck–drone routing in smart logistics: A comprehensive review

The surge in e-commerce and demand for same-day delivery have intensified the need for efficient last-mile logistics solutions. Traditional vehicle routing problems fall short in addressing real-world constraints such as traffic congestion, energy limitations, and dynamic delivery demands. In response, Multi-Agent Reinforcement Learning (MARL) has emerged as a transformative approach for optimizing truck–drone collaboration, enabling decentralized decision-making, real-time re-routing, and intelligent resource allocation.

This paper presents a comprehensive review of MARL-based truck–drone logistics, categorizing recent advancements in value-based, policy-based, and hybrid learning approaches, including Deep Q-learning, proximal policy optimization, and multi-agent deep deterministic policy gradient, and meta-reinforcement learning. It proposes a structured taxonomy based on learning strategies, optimization objectives, and environmental constraints. Key performance drivers such as delivery efficiency, energy-aware scheduling, and scalability are examined alongside real-world limitations, including traffic congestion, regulatory compliance, infrastructure dependence, and weather variability.

The paper also identifies critical challenges – computational complexity, communication overhead, and adversarial robustness and outlines future research directions, including hybrid learning architectures, blockchain-secured coordination, edge intelligence, and sustainability – driven MARL optimization. By synthesizing state-of-the-art research and identifying actionable pathways, this review provides foundational insights for advancing intelligent, adaptive, and eco-efficient truck–drone delivery systems.