Training SNNs Low Latency Utilizing Batch Normalization Through Time and Iterative Initialization Retraining

Abstract

Spiking Neural Network (SNN), developing on neuromorphic hardware, is a promising energy-efficient AI paradigm. However, processing over several timesteps reduces the energy benefits of SNNs due to high latency, the number of operations, and memory access costs from acquiring membrane potentials. Furthermore, their non-derivative nature makes SNNs difficult to train properly. To overcome these issues and leverage the full potential of SNNs, in this research, we offer a novel way for training deep SNNs utilizing Batch Normalization Through Time and Iterative Initialization and Retraining techniques. First, the BNTT improves low-latency and low-energy training in SNNs by allowing neurons to handle the spike rate over many timesteps. Second, we can obtain SNNs with up to unit latency pass during inference when applying the Iterative Initialization and Retraining technique during training SNNs. On the CIFAR-10, CIFAR-100, and ImageNet, we achieve cutting-edge SNN performance using a deep neural network with just one timestep. We achieve top-1 accuracy of 91.01%, 71.88%, and 69.8% on CIFAR-10, CIFAR-100, and ImageNet, respectively, using the VGG 16 architecture.