A Quantitative Analysis of Offensive Cyber Operation (OCO) Automation Tools

Abstract

The ecosystem for automated offensive security tools has grown in recent years. As more tools automate offensive security techniques via Artificial Intelligence (AI) and Machine Learning (ML), it may result in vulnerabilities due to adversarial attacks. Therefore, it is imperative that research is conducted to help understand the techniques used by these security tools. Our work explores the current state of the art in offensive security tools. First, we employ an abstract model that can be used to understand what phases of an Offensive Cyber Operation (OCO) can be automated. We then adopt a generalizable taxonomy, and apply it to automation tools (such as normal automation and the use of artificial intelligence in automation). We then curated a dataset of tools and research papers and quantitatively analyzed it. Our work resulted in a public dataset that includes analysis of (n=57) papers and OCO tools that are mapped to the the MITRE ATT&CK Framework enterprise techniques, applicable phases of our OCO model, and the details of the automation technique. The results show a need for a granular expansion on the ATT&CK Exploit Public-Facing application technique. A critical finding is that most OCO tools employed Simple Rule Based automation, hinting at a lucrative research opportunity for the use of Artificial Intelligence (AI) and Machine Learning (ML) in future OCO tooling.