Investigation of Weight Initialization Using Fibonacci Sequence on the Performance of Neural Networks*

Abstract

Initializing weights are important for fast convergence and performance improvement of Artificial Neural Network models. This study proposes a heuristic method to initialize weights for Neural Network with Fibonacci sequence. Experiments have been carried out with different network structures and datasets and results have been compared with other initialization techniques such as Zero, Random, Xavier and He. It has been observed that for small sized datasets, Fibonacci initialization technique reports 94% of test accuracy which is better than Random (85%) and close to Xavier (93%) and He (96%) initialization methods. Also, for medium sized dataset, we have noted that performance of Fibonacci weight initialization method is comparable with the same for Random, Xavier and He initialization techniques.