Reinforcement learning-based automated target motion analysis in underwater environments

Abstract

This study presents an automated target motion analysis (TMA) framework that leverages deep reinforcement learning (DRL) to enhance the accuracy and reliability of target state estimation from SONAR-derived bearing-only measurements in underwater environments. Traditional TMA methods-such as the manual 10-point divider and batch estimation-rely heavily on operator expertise and are susceptible to inaccuracies due to environmental noise and human error. To address these limitations, we employ a Proximal Policy Optimization (PPO)-based agent to automatically and robustly estimate the target speed. A customized TMA simulator was developed to generate diverse underwater scenarios, incorporating variations in target motion and noise levels to ensure the model’s generalization capability. The PPO agent learns to infer target speed directly from sequential bearing data, achieving a strong balance between exploration and exploitation. Experimental results demonstrate that the trained agent provides highly accurate and robust speed estimates, even under realistic noise conditions. This work contributes to the advancement of autonomous maritime surveillance and defense systems by significantly reducing human dependency and improving operational reliability.