FlowHash: Accelerating Audio Search With Balanced Hashing via Normalizing Flow

Abstract

Nearest neighbor search on context representation vectors is a formidable task due to challenges posed by high dimensionality, scalability issues, and potential noise within query vectors. Our novel approach leverages normalizing flow within a self-supervised learning framework to effectively tackle these challenges, specifically in the context of audio fingerprinting tasks. Audio fingerprinting systems incorporate two key components: audio encoding and indexing. The existing systems consider these components independently, resulting in suboptimal performance. Our approach optimizes the interplay between these components, facilitating the adaptation of vectors to the indexing structure. Additionally, we distribute vectors in the latent $\mathbb {R}^{K}$ space using normalizing flow, resulting in balanced $K$ -bit hash codes. This allows indexing vectors using a balanced hash table, where vectors are uniformly distributed across all possible $2^{K}$ hash buckets. This significantly accelerates retrieval, achieving speedups of up to 2× and 1.4× compared to the Locality-Sensitive Hashing (LSH) and Product Quantization (PQ), respectively. We empirically demonstrate that our system is scalable, highly effective, and efficient in identifying short audio queries ( $\leq$ 2 s), particularly at high noise and reverberation levels.