MetaMFL: Metamorphic Multiple Fault Localization Without Test Oracles

Multiple fault localization (MFL) identifies the positions of multiple faults (i.e., more than one fault) residing in a buggy program. It is notably more difficult as compared with single fault localization (SFL) which aims to locate a single fault (i.e., one fault) in a buggy program. Clustering-based multiple fault localization (CBMFL) is amongst the most popular MFL approaches, showing promising results in multiple fault localization. The requisite of launching CBMFL depends on test oracles to acquire the test results (i.e., a pass or a failure). In practice, test oracles are commonly not available known as the oracle problem, and CBMFL becomes infeasible in these cases. Inspired by metamorphic testing in solving the oracle problem, we attempt to combine this technique into CBMFL to broaden its application scope. Thus, we propose MetaMFL: Metamorphic Multiple Fault Localization, which leverages metamorphic testing to extend CBMFL to the cases where test oracles are not available. Specifically, MetaMFL uses metamorphic testing groups as minimum units of testing. It defines metamorphic features for representing those that have violated metamorphic relations. Using these features, CBMFL can perform clustering to support parallel debugging, thus achieving MFL without test oracles. The large-scale experiments show that MetaMFL largely retains the effectiveness of CBMFL even though test oracles are not available.