Sparsely connected autoencoder

Abstract

This work proposes to learn autoencoders with sparse connections. Prior studies on autoencoders enforced sparsity on the neuronal activity; these are different from our proposed approach - we learn sparse connections. Sparsity in connections helps in learning (and keeping) the important relations while trimming the irrelevant ones. We have tested the performance of our proposed method on two tasks - classification and denoising. For classification we have compared against stacked autneencoders, contractive autoencoders, deep belief network, sparse deep neural network and optimal brain damage neural network; the denoising performance was compared against denoising autoencoder and sparse (activity) autoencoder. In both the tasks our proposed method yields superior results.