On Classification of Acceptable Images for Reliable Artificial Intelligence Systems: A Case Study on Pedestrian Detection

Abstract

Images are essential data for many artificial intelligence (AI) systems such as pedestrian detection. However, image processing circuits or image storage devices may produce erroneous image data due to aging or radiation. In this paper we will show that there actually exists much tolerability in image errors. Moreover, for AI systems we find that the tolerability is even larger. This finding provides an attractive reliability enhancement solution by classifying and filtering acceptable images. This solution allows acceptable images to still go to the AI inference process, while unacceptable images are discarded, together with warning signals activated. In this work, we first evaluate and compare error tolerability of images from human and machine perspectives. Then a number of possible test methods to support machine based error-tolerance are discussed and compared in terms of their acceptability classification accuracy and computation cost. In particular, these methods should not need golden (error-free) images as the comparison basis. This greatly facilitates developing a low-cost on-line test architecture to enable a real-time reliability enhancement solution. Our experimental results show that when applying the suggested test method to pedestrian detection, 93.48% of the erroneous images can be correctly classified. The results also show that adopting machine-based error-tolerance can extend MTTF (Mean Time To Failure) of the pedestrian detection system up to additional 88.7%, while human vision based error-tolerance can extend only additional 35.1%.