An optimal transport-guided diffusion framework with mitigating mode mixture

Abstract

Diffusion probability models (DPMs) have achieved excellent results in image generation; however, their inference process is slow and tends to produce more mixed images. The autoencoder optimal transport (OT) model addresses the mode collapse/mixture problem from the OT perspective but produces low-quality images. Therefore, to generate high-quality images and mitigate mode mixture, we propose an innovative OT-guided diffusion framework. The key is to find the optimal truncation step $M$ to ensure that the class boundaries of the original data do not intersect during the forward process, ensuring that the generated image belongs to the same class as the initial point in the reverse process. The value of $M$ is determined by evaluating the Peak Signal-to-Noise Ratio, enabling us to mitigate the generation of mixed images. Specifically, our approach first involves embedding the images’ manifold into the latent space through an encoder. The images are subsequently decoded using latent codes, which are generated through an OT map from the Gaussian distribution to the empirical latent distribution. Finally, the trained $M$-step DPM is utilized to refine the image generated by the decoder. Experimental results demonstrate that our method not only improves image quality but also alleviates mode mixture in diffusion models. Additionally, it enhances sampling efficiency and reduces training cost compared to classical diffusion models.