A Network Connectivity-Aware Reinforcement Learning Method for Task Exploration and Allocation

Abstract

For a limited scale self-organized multi-agent system operating in environments with unknown task distributions, one challenge is to reduce the task response time via efficiently combining task exploration and allocation, another challenge is to improve the task completion rate via unlocking the potential of network cooperation in task allocation. However, in the existing studies, task allocation is generally regarded as an independent issue for known task distribution environments, rarely combined with task exploration, also hardly solving the conflict between the multi-hop network cooperation and mobility flexibility of agents. In view of this, this paper proposes a network connectivity-aware deep reinforcement learning method for task exploration and allocation in limited scale multi-agent systems (NCADRL4TEA). This method divides the task environment into regions and integrates task exploration with task allocation via two policies: a leaving policy to guide global task exploration among regions according to the distribution of agents and tasks, and a stay policy to guide local task allocation within each region according to the multi-hop network cooperation performance between agents. Further, in the stay policy, a network connectivity-aware task allocation optimization model is provided, which leads agents in the same region to cooperate with each other via multi-hop intermittent network connectivity and flexibly adjust their locations until the optimal multi-hop network cooperation performance is achieved. The experimental results verify that NCADRL4TEA can reduce the task response time in combination of task exploration and allocation, and improve the task completion rate in network cooperation.