A Low-cost Residue-based Scheme for Error-resiliency of RNN Accelerators

Abstract

Acceleration and power reduction requirements are usually the main constraints for the design of Artificial Neural Network (ANN) accelerators. However, in the case of safety-critical applications like autonomous driving, reliability takes precedence over other requirements. Although ANN algorithms provide a degree of inherent resiliency, the hardware part is still vulnerable to faults and may cause catastrophic failures. This paper proposes using residue codes for detecting soft errors in Recurrent Neural Networks (RNNs), and in particular, Long Short-Term Memory (LSTM) networks. We attach Concurrent Error Detection (CED) hardware units to an entire LSTM structure or its substructures. Depending on the granularity of the components to which they are applied, CEDs are referred to as coarse-grain or fine-grain CEDs. The simulation results show that in fault detection rate and misprediction coverage rate, fine-grain CEDs have a better performance than coarse-grain. Specifically, fine-grain residue-based CEDs provide up to 97% fault detection for extremely large (10-2) bit error rates. Moreover, they reduce the misprediction rate by 84% compared to unprotected LSTM.