A study on the role of latent variables in the encoder-decoder model using image datasets

Abstract

An encoder-decoder model consists of an encoder that encodes the input into a low-dimensional latent variable and a decoder that decodes the obtained latent variable to the same dimension as the input. The encoder-decoder model performs representation learning to automatically extract features of the data, but the model is a black box and it is not clear what features are extracted. We focused on whether including a skip connection between the encoder and decoder increased accuracy. It is generally believed that this skip connection plays a role in conveying high-resolution information. However, its actual role remains unclear. In this study, we focused on this concatenation. We experimentally clarified the role of the latent variables conveyed by this concatenation when the images given to the input and output were the same or different during training.