Kernel-Based Autoencoders for Large-Scale Representation Learning

Abstract

A primary challenge in kernel-based representation learning comes from the massive data and the excess noise feature. To breakthrough this challenge, this paper investigates a deep stacked autoencoder framework, named improved kernelized pseudoinverse learning autoencoders (IKPILAE), which extracts representation information from each building blocks. The IKPILAE consists of two core modules. The first module is used to extract random features from large-scale training data by the approximate kernel method. The second module is a typical pseudoinverse learning algorithm. To diminish the tendency of overfitting in neural networks, a weight decay regularization term is added to the loss function to learn a more generalized representation. Through numerical experiments on benchmark dataset, we demonstrate that IKPILAE outperforms state-of-the-art methods in the research of kernel-based representation learning.