Autonomous Security Analysis and Penetration Testing

Abstract

Security Assessment of large networks is a challenging task. Penetration testing (pentesting) is a method of analyzing the attack surface of a network to find security vulnerabilities. Current network pentesting techniques involve a combination of automated scanning tools and manual exploitation of security issues to identify possible threats in a network. The solution scales poorly on a large network. We propose an autonomous security analysis and penetration testing framework (ASAP) that creates a map of security threats and possible attack paths in the network using attack graphs. Our framework utilizes: (i) state of the art reinforcement learning algorithm based on Deep-Q Network (DQN) to identify optimal policy for performing pentesting testing, and (ii) incorporates domain-specific transition matrix and reward modeling to capture the importance of security vulnerabilities and difficulty inherent in exploiting them. ASAP framework generates autonomous attack plans and validates them against real-world networks. The attack plans are generalizable to complex enterprise network, and the framework scales well on a large network. Our empirical evaluation shows that ASAP identifies non-intuitive attack plans on an enterprise network. The DQN planning algorithm employed scales well on a large network $\sim 60 -70(\mathrm{s})$ for generating an attack plan for network with 300 hosts.