Understanding Linear Style Transfer Auto-Encoders

Abstract

Style transfer auto-encoder has recently been shown to be highly effective in synthesizing images with styles transferred from another image. In this work, we aim to provide an answer to this question by studying a simpler variant of STAE, namely, the linear style transfer auto-encoders (LinSTAEs), where the encoder and decoders are all linear models. We show that the objective function of LinSTAE, under the $\ell_{2}$ loss, affords a simple form, and the optimal solutions reveal the mechanism how the encoder capture joint characteristics from the input and the target domain, and the decoders restore their idiosyncrasies. We further show that at least for the linear case, the cycle reconstruction loss is not necessary - the vanilla LinSTAE objective function is already effective. We use numerical experiments on the synthetic and the MNIST dataset to showcase our findings.