Learning Robust Representations by Autoencoders With Dynamical Implicit Mapping

Abstract

Autoencoder is an unsupervised neural network that learns effective representations of data and has wide applications in feature learning, data compression, etc. However, Autoencoder is very sensitive to noise, resulting in low generalization and robustness of the model. To solve this problem, we propose a stable and efficient Autoencoder model called nmFunc-Autoencoder. Inspired by the Neural Memory Ordinary Differential Equation, the Neural Memory Activation Function uses its excellent dynamic nonlinear implicit mapping to establish a mapping relationship between external inputs and stable values to ensure the stability of distinguishable feature extraction, thereby performing better robustness when subjected to noise attacks. We conduct robustness experiments to evaluate its performance. The result showed that compared with other Autoencoder models, the data features extracted by the proposed model are more robust. Subsequently, in the execution efficiency experiments and ablation study, the model was shown to be low-cost and effective.