First Realization of Batch Normalization in Flash-Based Binary Neural Networks Using a Single Voltage Shifter

Abstract

Batch normalization (BN) is a technique used to enhance training speed and generalization performance by mitigating internal covariate shifts. However, implementing BN in hardware presents challenges due to the need for an additional complex circuit to normalize, scale and shift activations. We proposed a hardware binary neural network (BNN) system capable of BN in hardware, which is consist of an AND-type flash memory array as a synapse and a voltage sense amplifier (VSA) as a neuron. In this system, hardware BN was implemented using a voltage shifter by adjusting the threshold of the binary neuron. To validate the effectiveness of the proposed hardware-based BNN system, we fabricated a charge trap flash with a gate stack of SiO ₂ /Si ₃ N ₄ /SiO ₂ . The electrical characteristics were modelled by using BSIM3 model parameters so that the proposed circuit was successfully demonstrated by a SPICE simulation. Moreover, variation effects of the voltage shifter were also analyzed using Monte Carlo simulation. Finally, the performance of the proposed system was proved by incorporating the SPICE results into a high-level simulation of binary LeNet-5 for MNIST pattern recognition, resulting in the improvement of the proposed system in terms of power and area, compared to the previous studies.