Understanding and leveraging vocoder fingerprints for synthetic speech attribution

Abstract

With the rapid advancements in generative adversarial networks (GANs), neural vocoders have emerged as critical components for synthesizing intelligible speech. The rise of fake audio poses significant challenges and risks to national security due to malicious abuse. Although countermeasures have been proposed to detect deepfakes, attributing audio to specific vocoder architectures remains a challenging task. Existing approaches that directly input handcrafted features into sophisticated deep neural networks (DNNs) tend to neglect the misguidance of content-relevant features, which leads to poor generalization and efficacy. In this paper, we propose a novel framework that focuses on disentangling the vocoder fingerprint from audio to identify fake audio. To this end, we introduce an audio reconstructor based on the U-Net architecture that minimizes the preservation of the content-relevant features of the original audio. The residual between the raw and reconstructed latent vectors is then calculated to eliminate content-relevant features. The residual is finally fed into a classifier to determine the vocoder’s architecture. The extensive experiments demonstrate the effectiveness of our proposed method in attributing fake audio in various cross-test setups on large-scale datasets. Additionally, we apply our approach to binary fake audio detection and observe its remarkable generalizability even with unseen vocoders.