A Phish Scale: Rating Human Phishing Message Detection Difficulty

Abstract

As organizations continue to invest in phishing awareness training programs, many Chief Information Security Officers (CISOs) are concerned when their training exercise click rates are high or variable, as they must justify training budgets to those who question the efficacy of training when click rates are not declining. We argue that click rates should be expected to vary based on the difficulty of the phishing email for a target audience. Past research has shown that when the premise of a phishing email aligns with a user’s work context, it is much more challenging for users to detect a phish. Given this, we propose a Phish Scale, so CISOs and phishing training implementers can easily rate the difficulty of their phishing exercises and help explain associated click rates. We based our scale on past research in phishing cues and user context, and applied it to previously published data and new data from organization-wide phishing exercises targeting approximately 5 000 employees. The Phish Scale performed well with the current phishing dataset, but future work is needed to validate it with a larger variety of phishing emails. The Phish Scale shows great promise as a tool to help frame data sharing on phishing exercise click rates across sectors. Keywords—phishing cues, embedded phishing awareness training, operational data, network security, phishing defenses, security defenses