HNSG – A SNN Training Method Ultilizing Hidden Network

Abstract

Spiking Neural Network is more energy efficient compared to traditional ANNs, and many training methods of SNNs have been proposed in past decades. However, traditional backward-propagation based training methods are difficult to deploy on SNN due to its discontinuous gradient. Previous works mainly focused on weight training or weight transferring. The Hidden Network inspired by Lottery Ticket Hypothesis that is proposed for convolutional neural networks opens possibility of network connection training on SNN. In this article, a training algorithm based on Hidden Network is applied to SNN to show its potential on neuromorphic spiking networks. A novel training method called HNSG is proposed that modifies hidden network search using surrogate gradient function based back propagation. The proposed HNSG method is tested on image classification task using MNIST with simple two fully-connected layer SNN model. Simulation shows HNSG reaches 93.73% accuracy on average fire intensity of 0.138 with LIF neuron.