The True Cost of Errors in Emerging Memory Devices: A Worst-Case Analysis of Device Errors in IMC for Safety-Critical Applications

Abstract

In-memory computing devices are prone to errors that can significantly affect the accuracy of neural network inference. While average accuracy loss is often used to evaluate the impact of such errors, this metric may not be reliable for safety-critical systems where worst-case performance is crucial. In this work, we present a comprehensive statistical analysis of the variability in the accuracy of quantized neural networks. We conduct experiments on two well-known neural network architectures, LeNet-5 and ResNet20, using both 4-bit and 8- bit quantization, and measure the worst-case impact of errors on model accuracy. Our results demonstrate that worst-case variation is much more significant than the impact on average accuracy and that 8-bit quantization is more susceptible to errors. We also investigate the potential of intra-layer mixed error injection to mitigate the effects of errors and show that it can improve the worst-case accuracy of neural networks.