Neural predictor aided policy optimization for adversarial controlled sensing

Abstract

This paper is concerned with the fundamental problem of controlled sensing, namely how to optimize signal processing resources in a sensor network in order to detect the true hidden state of the environment, when the sensors are subject to adversarial attacks. The sensing task is performed by a legitimate agent who actively selects observations generated by a set of sensors and makes inference about the true state by minimizing the error probability. The adversary may have access to all or a subset of the sensors and can influence the quality of observations. Agents may have only partial access to the complete data set, leading to different beliefs about the true state, different perceptions of the error probability. To address the complexities of this problem, we define well motivated approximate structures that fill the gap of partial information. We provide three different objective functions for training a neural predictor, and we demonstrate how prediction quality is a precondition for detection performance. Based on the above concepts, we propose a novel deep reinforcement learning (DRL) algorithm, termed Predictive Proximal Policy Optimization for Adversarial Controlled Sensing (3POACS) algorithm. This algorithm combines building blocks from single agent DRL, problem specific reward reshaping procedures, and a neural predictor. Finally, we use an anomaly detection example to demonstrate the superiority of the proposed method over previous non-adversarial approaches. Experiments show that the new algorithm favorably competes with DRL algorithms with access to oracle predictors.