UBAL

Abstract

Artificial neural networks, in particular the deep end-to-end architectures trained by error backpropagation (BP), are currently the topmost used learning systems. However, learning in such systems is only loosely inspired by the actual neural mechanisms. Algorithms based on local activation differences were designed as a biologically plausible alternative to BP. We propose Universal Bidirectional Activation-based Learning, a novel neural model derived from contrastive Hebbian learning. Similarly to what is assumed about learning in the brain, our model defines a single learning rule that can perform multiple ways of learning via special hyperparameters. Unlike others, our model consists of mutually dependent, yet separate weight matrices for different directions of activation propagation. We show that UBAL can learn different tasks (such as pattern retrieval, denoising, or classification) with different setups of the learning hyperparameters. We also demonstrate the performance of our algorithm on a machine learning benchmark (MNIST). The experimental results presented in this paper confirm that UBAL is comparable with a basic version BP-trained multilayer network and the related biologically-motivated models.