Integrating Multi-Agent Reinforcement Learning and XGBoost for Efficient Large-Scale Dynamic Workflow Scheduling in Cloud Environments

Abstract

Workflow scheduling in heterogeneous cloud environments must operate under dynamic and heterogeneous conditions. Heuristic algorithms cannot adapt online to changing conditions, while single-agent deep reinforcement learning (DRL) faces issues such as exponential growth in the action space, excessive training time, and limited interpretability. This paper proposes MAR-XGB, a hybrid scheduling framework that combines multi-agent reinforcement learning (MARL) with an XGBoost predictor. The tree ensemble estimates task runtime and execution cost, from which an urgency score is derived. Each agent uses lightweight Monte Carlo Tree Search (MCTS) to select VM locations and make online placement decisions over the Top-k tasks, ensuring real-time adaptability while reducing action-space complexity and improving prediction transparency and decision auditability. Experiments on representative WfCommons scientific workflows and controlled high-load DAG scenarios show that our approach outperforms traditional and state-of-the-art schedulers in terms of makespan, resource utilization, and cost efficiency. As the complexity and scale of workflows increase, our approach demonstrates improved scalability, indicating its potential to handle dynamic cloud workloads with fluctuating demand.