Autoencoder performance analysis with adaptive and trainable activation function to compress images

Activation functions are essential keys to good performance in a neural network. Many functions can be used, and the choice of which one to use depends on the issues addressed. New adaptable and trainable activation functions have been studied lately, which are used to increase network performance. This study intends to evaluate the performance of an artificial neuron that uses adaptive and trainable functions in an Autoencoder network for image compression problems. The tested neuron, known as Global-Local Neuron, comprises two complementary components, one with global characteristics and the other with local characteristics. The global component is given by a sine function and the local component by the hyperbolic tangent function. The experiment was carried out in two stages. In the first one, different activation functions, GLN, Tanh, and Sine, were tested in an MLP-type Autoencoder neural network model. Different compression ratios were considered when varying the size of the Autoencoder bottleneck layer, and 48 samples were obtained for each value of this layer. The metrics used for the evaluation were the loss value obtained in the test set and the number of epochs necessary to reach a stopping criterion. In the second step, the classification accuracy of the images compressed by the encoder block of the previous model was evaluated, using a Wide Residual Networks (WRN) network and the Support Vector Machines (SVM) method. The results obtained indicated that the use of Global-Local Neuron improved the network training speed, obtained better classification accuracy for compression up to 50% in a WRN network, and proved the adaptability in image classification problems.