ODE-based generative modeling: Learning from a single natural image

Abstract

Single image generation aims to learn the internal statistical distribution from a single natural image to generate diverse samples of arbitrary scales, serving as a tool for image manipulation tasks. Existing methods adopt the same pyramid structure for both training and multi-stage sampling to ensure the stability of the generation model. However, these methods result in a large number of sampling time steps and extra noise at each level of the pyramid to sample a single image. In this work, we propose a Single image generative model based on Ordinary Differential Equation (ODE), dubbed as SinODE. Instead of relying on a repetitive multi-stage sampling process, SinODE reformulates single image sampling as a unified integration framework, reducing sampling times while eliminating unnecessary noise injection. To that end, we build straight paths connecting Gaussian noise to scaled images and generating samples with a multiple piece-wise integration mechanism. Furthermore, our method can employ external text to control the direction of generation, producing personalized new content or style without requiring model fine-tuning. SinODE can also be effortlessly applied to other image manipulation tasks, such as image style transfer and harmonization. Extensive experiments demonstrate that SinODE surpasses current state-of-the-art methods, producing high-quality samples with exceptional diversity.