Efficient adaptive test case selection for DNNs robustness enhancement

Abstract

Deep neural networks (DNNs) have been widely used in various fields, and testing for DNN-based software has become increasingly important. To discover potential faults in DNNs, a large number of test cases and their corresponding labels are required. However, labeling so many test cases consumes enormous costs. Although there have been many test case selection techniques for DNN models, these techniques still have problems such as high overhead, low efficiency, and poor diversity. To address this problem, this paper proposes an efficient adaptive test case selection method based on the principle of uniform distribution of test cases called EATS. Based on the idea of adaptive testing, EATS combines the uncertainty of the model and the diversity of faults to calculate the distance of test cases and sort them, then gives priority to test cases with a higher probability of causing faults. We conduct experiments on four popular datasets and four representative DNN models. Experiment results show that, compared with the existing eight methods, EATS performs better in uniformity of test case distribution, diversity of errors found, model optimization, and optimization efficiency.