Accelerated Local Training of CNNs by Optimized Direct Feedback Alignment Based on Stochasticity of 4 Mb C-doped Ge2Sb2Te5 PCM Chip in 40 nm Node

Abstract

On-chip local training is highly desirable for the application of deep neural networks in environment-adaptive edge platforms, which however is hindered by the high time and energy costs of training. Here, we demonstrate efficient training of VGG-16 and LeNet-5 by optimized direct feedback alignment that replaces the layer-by-layer back propagation (BP) of errors. For the first time, the inherent stochasticity in phase change memory fabricated in 40 nm node is exploited to build a merged random feedback matrix with reduced hardware cost. Due to the physical generation of merged matrix and in-memory error computing as well as proposed conductance drift (CD) compensation protocols, the training time and energy consumptions of VGG-16 are reduced by 3× and 3.3×, respectively, compared with hardware-accelerated in-memory BP training, with 90% accuracy on CIFAR-10.