Energy-efficient DNN-training with Stretchable DRAM Refresh Controller and Critical-bit Protection

Training DNN is a time-consuming process and requires intensive memory resources. Many software-based approaches were proposed to improve the performance and energy efficiency of inferring DNNs. Meanwhile training hardware is still received limited attention. In this work, we present a novel DRAM architecture with critical-bit protection. Our method targets main memory and graphical memory of the training system. Experimented on GEM5-GPGPUsim, our proposed DRAM architecture can achieve 23% and 12% DRAM energy reduction with floating point 32bit on main and graphical memories, respectively. Also, it further improves system's performance by 0.43˜ 4.12% while maintaining a negligible accuracy drops in training DNNs.