GAN-Based Hybrid Sampling Method for Transaction Fraud Detection

In the digital era, effective Transaction Fraud Detection (TFD) is essential to ensuring financial security. The considerable class imbalance, with legitimate transactions vastly outnumbering fraudulent ones, presents a significant challenge for TFD models to accurately identify fraudulent patterns. While existing sample-balancing strategies address class imbalance effectively in many contexts, they often fall short in TFD due to fraudsters’ sophisticated concealment tactics, which lead to pronounced behavioral overlap between fraudulent and legitimate transactions. In this paper, we introduce a novel Generative Adversarial Network-based Hybrid Sampling method (GANHS) to effectively address the class imbalance issue. GANHS employs a dual-discriminator generative adversarial network to generate synthetic samples that accurately reflect the characteristics of fraudulent activity, while an adaptive neighborhood-based undersampling technique refines these samples to minimize overlap with legitimate ones. This hybrid approach not only enhances the model’s ability to learn fraud patterns by generating high-quality samples but also improves its resilience against highly concealed fraudulent activities. Experiments on real-world and public datasets demonstrate that GANHS outperforms its competitive peers, with gains of 0.5%–8.7% in average $F_{1}$-Score and 1.0%–7.0% in G-mean, highlighting its strong potential for improving the reliability and effectiveness of TFD systems in complex, high-risk financial scenarios.