Multi-UAV maritime collaborative behavior modeling based on hierarchical deep reinforcement learning and DoDAF process mining

Abstract

Autonomous systems, particularly in multi-UAV maritime operations, are becoming increasingly complex, posing significant challenges to dynamically modeling based on traditional systems engineering modeling methods. This paper proposes an innovative data-driven approach that combines deep reinforcement learning and process mining with Department of Defense Architecture Framework (DoDAF) views to learn and extract dynamic multi-UAV collaborative behaviors. First, a hierarchical multi-agent reinforcement learning framework is developed to simulate high-value complex maritime UAV collaboration, where agents learn implicit high-level task selection patterns while executing predefined low-level behaviors. Then, a DoDAF-oriented process mining algorithm is designed, which is the key innovation, to automatically extract DoDAF operational view-5b diagrams from learned behavioral pattern data. The experimental validation demonstrates this method excels at systematically extracting dynamic multi-UAV collaborative behaviors. The proposed approach could effectively bridge the gap between AI-based implicit behavior pattern learning and system engineering-based explicit behavior modeling requirement, contributing to the development of interpretable autonomous system and discovering effective collaborative behavior tactics.