LMs go Phishing: Adapting Pre-trained Language Models to Detect Phishing Emails

Abstract

Despite decades of research, the problem of Phishing in everyday email communication is ever so prevalent. Traditionally viewed as a text-classification task, the task of phishing detection is an active defense against phishing attempts. Mean-while, progress in natural language processing has established the universal usefulness of adapting pre-trained language models to perform downstream tasks, in a paradigm known as pre-train-then-fine-tune. In this work, we build on this paradigm, and propose two language models that are adapted on 725k emails containing phishing and legitimate messages. We use these two models in two ways: 1) by performing classification-based fine-tuning, and 2) by developing a simple priming-based approach. Our approaches achieve empirical gains over a good deal of prior work, achieving near perfect performance on in-domain data, and relative improvements on out-of-domain emails.