Non-Equilibrium Learning and Cyber-Physical Security

This paper introduces a framework for non-equilibrium behavior analysis in cyber-physical systems for security purposes. To categorize the player, we employ the principles of reinforcement learning in order to derive an iterative method of optimal responses that determine the policy of an agent with level-$k$ intelligence in a general non-zerosum, nonlinear environment. For the special case of zero-sum, linear quadratic games we derive appropriate non-equilibrium game Riccati equations. To obviate the need for complete knowledge of the system dynamics, we employ a Q-learning algorithm as a best response solver. We then design an estimator that determines the distribution of intelligence levels in the adversarial environment of the system. Finally, simulation results showcase the efficacy of our approach.