CCM-FCC: LLM-powered cognition-centered AI agent framework for proactive human-robot collaboration

Abstract

Proactive human-robot collaboration (PHRC) primarily relies on predefined rule-based integration of perception, analysis and decision-making into a unified framework, which limits its autonomy and interactivity in dynamic scenarios such as disassembly and assembly. Although AI agent equipped with memory and interaction functions exhibits enhanced adaptability, their task-specific designs result in a lack of holistic cognition, thereby limiting their generalization capability. This paper proposes a Large Language Model (LLM)-powered cognition-centered AI agent framework, which addresses these challenges through the “Cognitive Core Management–Functional Cluster Collaboration” (CCM-FCC) paradigm. Specifically, to enhance the generalization capability of the AI agent, we developed a semantic Chain-of-Thought (CoT) prompt learning-driven cognitive core for predicting key task factors. The semantic CoT prompt learning, which couples task semantics with reasoning logic, empowers the pre-trained LLM to improve the key factors prediction. Subsequently, to ensure centralized management of the cognitive core, we designed a dual-dimensional feature-constrained functional activation module. It extracts task semantic cues from the key factors and autonomously activates functional modules within the AI agent, constrained by task complexity and operator state. Furthermore, a task-semantic-driven functional cluster collaboration module is proposed to generate the optimal collaboration strategy. Finally, a deep reinforcement learning model is constructed to enable the robot to proactively collaborate with the operator for PHRC. The experiments on HRC tasks demonstrates the effectiveness of the proposed method.