Impact of anti-phishing tool performance on attack success rates

Abstract

Phishing website-based attacks continue to present significant problems for individual and enterprise-level security, including identity theft, malware, and viruses. While the performance of anti-phishing tools has improved considerably, it is unclear how effective such tools are at protecting users. In this study, an experiment involving over 400 participants was used to evaluate the impact of anti-phishing tools' accuracy on users' ability to avoid phishing threats. Each of the participants was given either a high accuracy (90%) or low accuracy (60%) tool and asked to make various decisions about several legitimate and phishing websites. Experiment results revealed that participants using the high accuracy anti-phishing tool significantly outperformed those using the less accurate tool in their ability to: (1) differentiate legitimate websites from phish; (2) avoid visiting phishing websites; and (3) avoid transacting with phishing websites. However, even users of the high accuracy tool often disregarded its correct recommendations, resulting in users' phish detection rates that were approximately 15% lower than those of the anti-phishing tool used. Consequently, on average, participants visited between 74% and 83% of the phishing websites and were willing to transact with as many as 25% of the phishing websites. Anti-phishing tools were also less effective against one particular type of threat. The results suggest that while the accuracy of anti-phishing tools is a critical factor, reducing the success rates of phishing attacks requires other considerations such as improving tool interface/warning design and enhancing users' knowledge of phishing. Given the prevalence of phishing-based web fraud, the findings have important implications for individual and enterprise security.