Deep reinforcement learning and ant colony optimization supporting multi-UGV path planning and task assignment in 3D environments

Abstract

With the development of artificial intelligence, the application of unmanned ground vehicles (UGV) in outdoor hazardous scenarios has received more attention. However, the terrains in these environments are often complex and undulating, which also pose higher challenges to the multi-UGV path planning and task assignment (MUPPTA) optimization. To efficiently improve the multi-UGV collaboration in 3D environments, a MUPPTA method is proposed based on double deep Q learning network (DDQN) and ant colony optimization (ACO) to jointly optimize the path planning and task assignment decisions of multiple UGVs. The authors first comprehensively consider the characteristics of the 3D environments, and model the MUPPTA problem as a combinatorial optimization problem. To tackle it, the original problem is decomposed into the multi-UGV path planning sub-problem and task assignment sub-problem, and solve them separately. First, the path planning sub-problem in the 3D environments is transformed into a Markov decision process (MDP) model, and a multi-UGV path planning algorithm based on DDQN (MUPP-DDQN) is proposed to obtain the optimal paths and actual path costs between tasks through extensive offline learning and training. Based on this, a multi-UGV task assignment algorithm is further proposed based on ACO (MUTA-ACO) to solve the task assignment sub-problem and achieve the optimal task assignment solution. Simulation results show that the proposed method is more cost-effective and time-saving compared to other comparison algorithms.