Identify Coincidental Correct Test Cases Based on Fuzzy Classification

Abstract

In software testing, Coincidental Correct (CC) test case, which implement the faulty statement but with a correct output, has been investigated with a negative effects on coverage-based fault localization. Coincidental correct test case identification and manipulation had been studied and many identification methods are proposed, in which clustering based method is widely used. In this paper, a machine learning based fuzzy classification technique is proposed. We first present an approach to identify truly CC test cases for single fault version programs. Then KNN algorithm is adopted to classify the remaining passed test cases and three types of modified fuzzy suspiciousness metrics are presented based on three proposed CC test cases manipulation strategies. Empirical studies are conducted on 102 faulty versions of six programs, and the results indicate that the proposed approach makes the recall and false positive of CC test cases are 82% and 5% in average. In addition, the proposed fuzzy CC test cases manipulation strategies can improve the effectiveness of fault localization.