An Examination of Industry Standards of Success within Penetration Testing Groups

Penetration testing groups can be used as an ethical proxy to study cybercrime groups, as both parties share the common goal of identifying and exploiting weaknesses in their targets’ systems. Pentesters often use existing industry standards to guide their performance and practices, but little research has investigated how these standards operate in simulated cybersecurity exercises. Using the experiences of college students in the 2018 and 2019 National Collegiate Penetration Testing Competition (CPTC), a simulation of a professional real-world penetration test, this study seeks to further examine pentesting metrics. Metrics from industry standards of pentesting practices are compared to the metrics identified by the CPTC participants, revealed through semi-structured group interviews. Industry metrics include standards, such as methods, information gathering, attack generation, quantity of findings, quality of findings, and reporting of findings. Other additional metrics identified by the CPTC participants include skills of the team, the environment, expectations, and the relationships among group members. This study uses a qualitative methodological approach to examine the metrics of success identified by pentesters as they reflect on their decisions, actions, and performance.