Audio Deepfake Detection Using Deep Learning

Abstract

This study introduces an enhanced Siamese convolutional neural network (Siamese CNN) architecture with a novel StacLoss function and self-attention modules for efficient identification of audio deepfakes. Our module directly compares unprocessed original audio with modified audio by initially applying convolutional operations and dual branches to extract complex characteristics from raw audio signals. These operations are followed by residual connections, which enhance the network's performance. The self-attention modules are trained in a layered way alongside these fundamental layers to detect multi-headed attention within audio frames. The StacLoss output represents a customized version of the contrastive loss function. It aids the network in distinguishing between original and modified audios by minimizing the loss between pairs of original audio that have the same identity while maximizing the distance between manipulated audio samples and enhances the process of extracting features compared to standard techniques. The efficacy of the method has been verified by examining a range of audio modifications, and its resilience has been thoroughly assessed on the ASVspoof2019 dataset by comprehensive testing across all possible audio manipulation situations. The proposed Siamese convolutional neural network (CNN) outperformed both machine and deep learning models, achieving impressive metrics. It achieved a remarkable accuracy of 98%, precision of 97%, recall of 96%, F1 score of 96.5%, ROC-AUC of 99%, and an equal error rate (EER) of 2.95%.