Phishing email detection using vector similarity search leveraging transformer-based word embedding

Abstract

As cybercrime increases, using email cautiously is crucial. Phishing emails are a major threat, often exploited to steal sensitive data and cause financial losses. While anti-phishing techniques exist, evolving phishing tactics make countering them challenging. This study proposes a phishing detection system using transformer-based word embedding and vector similarity search. Pre-trained models like Dense Passage Retrieval (DPR) create high-dimensional vector embeddings from emails, stored in a vector database for real-time similarity searches. The proposed approach outperforms traditional machine learning by automating feature extraction and improving similarity search efficiency, making it more effective in detecting phishing emails. Empirical evaluation has been conducted using three publicly available datasets Enron, Nazario phishing corpora, and the Phishing validation emails dataset. The system demonstrates the superior performance, achieving 98.43% accuracy, 98.44% precision, 98.38% recall, 98.41% F1-score, and an area under the curves (AUC) of 0.984 using cosine similarity.