A Distance-Based Dynamic Random Testing Strategy for Natural Language Processing DNN Models

Abstract

Deep neural networks (DNNs) have achieved tremendous development while they may encounter with incorrect behaviors and result in economic losses. Identifying the most represented data become critical for revealing incorrect behaviours and improving the quality DNN-driven systems. Various testing strategies for DNNs have been proposed. However, DNN testing is still at early stage and existing strategies might not sufficiently effective. Dynamic random testing (DRT) strategy uses the feedback mechanism to guide the test case selection, which has been proved to be effective in fault detection. However, its efficacy for Natural Language Processing (NLP) DNN models has not been thoroughly studied. In this paper, a Distance-based DRT with prioritization (D-DRT-P) is proposed, which combines the priority information and distance information into DRT to guide the selection of test cases and testing profile adjustment. Empirical studies demonstrate that D-DRT-P can improve the fault detecting effectiveness than other test prioritization strategies in most cases.