A Comparative Analysis of Activation Function, Evaluating their Accuracy and Efficiency when Applied to Miscellaneous Datasets

Abstract

Numerous deep learning architectures have been developed as a result of activation functions (AFs), which are crucial for allowing deep neural networks to deal with intricate real-world problems. In order to achieve cutting-edge performance, AFs play a crucial role by facilitating diverse computations between the hidden and output layers. This paper presents a comparison between various activation function like sigmoid, tanh, ReLU, Softmax on thedatasetMNIST, CIFAR-10 and IRIS and their accuracy on these datasets with minimum errors. These observations offer valuable insights into determining the most suitable activation function for diverse scenarios and datasets, thereby providing a comprehensive understanding of the optimal activation function for distinct situations.