TALKER: A Task-Activated Language Model Based Knowledge-Extension Reasoning System

Abstract

Training drones to execute complex collective tasks via multi-agent reinforcement learning presents significant challenges. To address these challenges, this letter introduces the Task-Activated Language model-based Knowledge-Extension Reasoning system. Specifically, we trained drones in two fine-grained skills and developed an action primitive library based on these capabilities, enabling a hierarchical approach to managing complex swarm operations. Leveraging this primitive library, we employ large language models to perform task planning, continuously refining the planning outcomes based on external user feedback. Successful task codes are temporarily stored within the action primitive library, with their utilization being authorized based on internal feedback from maintainers. We defined this process as knowledge expansion. In addition, more refined customized prompts are generated based on task descriptions and the action primitive documentation, a mechanism referred to as Task Activation. Our system synergistically integrates task activation and knowledge expansion mechanisms, enabling continuous evolution through human feedback to effectively manage extensive swarms in the execution of complex collective tasks. Experimental results demonstrate the superior performance of our system in various drone swarm tasks, including collaborative search, object tracking, cooperative interception, and aerial patrol.