Train Offline, Refine Online: Improving Cognitive Tracking Radar Performance With Approximate Policy Iteration and Deep Neural Networks

Abstract

A cognitive tracking radar continuously acquires, stores, and exploits knowledge from its target environment in order to improve kinematic tracking performance. In this work, we apply a reinforcement learning (RL) technique, API-DNN, based on approximate policy iteration (API) with a deep neural network (DNN) policy to cognitive radar tracking. API-DNN iteratively improves upon an initial base policy using repeated application of rollout and supervised learning. This approach can appropriately balance online versus offline computation in order to improve efficiency and can adapt to changes in problem specification through online replanning. Prior state-of-the-art cognitive radar tracking approaches either rely on sophisticated search procedures with heuristics and carefully selected hyperparameters or deep RL (DRL) agents based on exotic DNN architectures with poorly understood performance guarantees. API-DNN, instead, is based on well-known principles of rollout, Monte Carlo simulation, and basic DNN function approximation. We demonstrate the effectiveness of API-DNN in cognitive radar simulations based on a standard maneuvering target tracking benchmark scenario. We also show how API-DNN can implement online replanning with updated target information.