Modified Neural Network Activation Function

Abstract

Neural Network is said to emulate the brain, though, its processing is not quite how the biological brain really works. The Neural Network has witnessed significant improvement since 1943 to date. However, modifications on the Neural Network mainly focus on the structure itself, not the activation function despite the critical role of activation function in the performance of the Neural Network. In this paper, we present the modification of Neural Network activation function to improve the performance of the Neural Network. The theoretical background of the modification, including mathematical proof is fully described in the paper. The modified activation function is code name as SigHyper. The performance of SigHyper was evaluated against state of the art activation function on the crude oil price dataset. Results suggested that the proposed SigHyper was found to improved accuracy of the Neural Network. Analysis of variance showed that the accuracy of the SigHyper is significant. It was established that the SigHyper require further improvement. The activation function proposed in this research has added to the activation functions already discussed in the literature. The study may motivate researchers to further modify activation functions, hence, improve the performance of the Neural Network.