Empirical Activation Function Effects on Unsupervised Convolutional LSTM Learning

Abstract

This paper empirically evaluates and analyzes the effect of the choice of recurrent activation and unit activation functions on the unsupervised convolutional LSTM learning process. The goal of this work is to provide guidance for selecting the optimal non-linear activation function for the convolutional LSTM models which target the video prediction problem. This paper shows an empirical analysis of different non-linear activation functions that are commonly implemented in different deep learning APIs. We used the moving MNIST dataset as the most common benchmark for video prediction problems.