DeFinder: Error-sensitive testing of deep neural networks via vulnerability interpretation

Abstract

DNN testing evaluates the vulnerability of neural networks through adversarial test cases. The developers implement minor perturbations to the seed inputs to generate test cases, which are guided by meticulously designed testing criteria. Nevertheless, current coverage-guided testing methods rely on covering model states rather than analyzing the influence of seed inputs on inducing erroneous behaviors. In this paper, we propose a novel DNN testing method called DeFinder, which generates error-sensitive tests by implementing an explainable framework for neural networks to establish correlations between model vulnerability and seed inputs. By systematically analyzing vulnerable regions within seed inputs, DeFinder significantly improves the test suite’s ability to maximize test coverage and expose errors. To validate the effectiveness of DeFinder, we conduct comprehensive experiments with nine deep neural network models from two popular computer vision datasets. We compare the proposed method with several state-of-the-art DNN testing tools. The experimental results demonstrate that DeFinder improves the error-triggering ratio by up to 58% and increases test coverage by up to 4.3%. For reproducibility, the artifact for this work is available at public repository: https://github.com/Konatazz/DeFinder.