TransStyle: Transformer-based StyleGAN for image inversion and editing

Abstract

Image inversion using StyleGAN retrieves latent codes by embedding real images into the GAN’s latent space, enabling attribute editing and high-quality image generation. However, existing methods often struggle with reconstruction reliability and flexible editing, resulting in low-quality outcomes. To address these issues, we propose TransStyle, a new StyleGAN inversion model based on Transformer technology. Our model features a novel encoder structure, PACP (Path Aggregation with Covariance Pooling), for improved feature representation and a feature prediction head that uses covariance pooling. Additionally, we propose a Transformer-based module to enhance interactions with semantic information in the latent space. StyleGAN then uses this enhanced latent code to generate images with high fidelity and strong editability. Experimental results demonstrate that our method achieves at least 5% higher face reconstruction similarity compared to current state-of-the-art techniques, confirming the advantages of TransStyle in image reconstruction and editing quality.