How well does GPT phish people? An investigation involving cognitive biases and feedback

Phishing scams have increased drastically over the years. Prior research has investigated various ways to prevent phishing email scams. However, little is known about human decisions against phishing emails that contain cognitive biases and are either crafted by humans or large-language models (LLMs). Also, less is known about how humans can be trained against such emails. This research aimed to address this literature gap by investigating the effectiveness of human-crafted phishing emails versus GPT3 crafted phishing emails (GPT-3 being an LLM). The study consisted of two between-subjects conditions (N = 30 per condition): human and GPT. Each condition contained three rounds with a total of 40 trials, and participants were required to mark the degree to which the presented email was genuine or phishing in each trial. The second round provided feedback to participants in both conditions. The results showed that human-crafted emails were more effective in phishing people compared to GPT-3 crafted emails even after training across different cognitive biases. However, humans felt more confident against human-crafted emails compared to GPT-3 crafted emails. We highlight the implications of these results for LLM crafted phishing attacks compared to human-crafted phishing attacks.