On the Correlation between the Effectiveness of Metamorphic Relations and Dissimilarities of Test Case Executions

Abstract

Metamorphic testing (MT) is a property-based automated software testing method. It alleviates the oracle problem by testing programs against metamorphic relations (MRs), which are necessary properties among multiple executions of the target program. For a given problem, usually more than one MR can be identified. It is therefore of practical importance for testers to know the nature of good MRs, that is, which MRs are likely to have higher chances of revealing failures. To address this issue we investigate the correlation between the fault-detection effectiveness of MRs and the dissimilarity (distance) of test case execution profiles. Empirical study results reveal that there is a strong and statistically significant positive correlation between the fault-detection effectiveness and the distance. The findings of this research can help to develop automated means of selecting/prioritizing MRs for cost-effective metamorphic testing.