Evaluation of Strategic Decision taken by Autonomous Agent using Explainable AI

Abstract

Autonomous intrusion detection systems assess the data intelligently and take strategic decision to detect and mitigate cyber-attacks. These decisions have to be explained and evaluated for the transparency and correctness. Explainable Artificial Intelligent (XAI) methods that explore how features contribute or influence a decision taken using an algorithm can be useful for the purpose. XAI method of Testing with Concept Activation Vectors (TCAV) has been used recently to show the importance of high level concepts for a prediction class in order to deliver explanations in the way humans communicate with each other. This work explores the possibility of using TCAV to evaluate the strategic decision made by autonomous agents. A case study in the context of DoS attack is analysed to show that TCAV scores for various DoS attack classes and normal class of KDD99 data set can be used to evaluate the strategic decisions. The proposed method of analysis provides a quantifiable method to justify the current strategy or change in the strategy if required.