Clustering Diffusion Model with Frequency-Signal Modulation for Variational Graph Autoencoders.

Abstract

Variational autoencoders (VAEs) have been widely used for node clustering, with existing methods mainly focusing on enhancing the expressiveness of their latent space. Recently, the integration of diffusion models with VAEs has provided new opportunities to achieve this objective. However, the mechanism by which the diffusion model improves performance remains unclear. To bridge this gap, we conduct an empirical analysis from the perspective of graph spectral theory, revealing that the signal modulation induced by diffusion models closely aligns with the low-frequency spectral characteristics of VAEs, which in turn explains their effectiveness. Nevertheless, further experiments highlight that diffusion models exhibit limitations in modulating high-frequency signals, which diverge from the spectral characteristics of VAEs. Moreover, existing diffusion methods fail to enable the latent space to adequately capture and reflect cluster-specific characteristics. To address these challenges, we propose a novel plug-and-play method, FVD, to improve the performance of VAE-based methods in node clustering tasks. Specifically, we incorporate the graph wavelet transform as a secondary signal modulator, enabling independent adjustments of specific frequency bands to better align with the spectral characteristics of VAEs. Additionally, we introduce the Student's t-distribution as a conditional constraint in the reverse process of FVD, deriving a more compact variational lower bound. This enhancement preserves fine-grained node information while focusing on clustering details, effectively mitigating the cluster collapse phenomenon. Comprehensive experimental results demonstrate that integrating FVD with existing methods achieves competitive performance improvements in most cases.