VAE-GNA: a variational autoencoder with Gaussian neurons in the latent space and attention mechanisms

Abstract

Variational autoencoders (VAEs) are generative models known for learning compact and continuous latent representations of data. While they have proven effective in various applications, using latent representations for classification tasks presents challenges. Typically, a straightforward approach involves concatenating the mean and variance vectors and inputting them into a shallow neural network. In this paper, we introduce a novel approach for variational autoencoders, named VAE-GNA, which integrates Gaussian neurons into the latent space along with attention mechanisms. These neurons directly process mean and variance values through a suitable modified sigmoid function, not only improving classification, but also optimizing the training of the VAE in extracting features, in synergy with the classification network. Additionally, we investigate both additive and multiplicative attention mechanisms to enhance the model’s capabilities. We applied the proposed method to automatic cancer detection using near-infrared (NIR) spectral data, showing that the experimental results of VAE-GNA surpass established baselines for spectral datasets. The results obtained indicate the feasibility and effectiveness of our approach.