MUT Model: a metric for characterizing metamorphic relations diversity

Abstract

Metamorphic testing emerged as a solution to the Oracle problem, with its foundation deeply rooted in the concept of Metamorphic Relations (MRs). Researchers have made an intriguing discovery that certain diverse MRs exhibit similar fault detection capabilities as the test oracle. However, defining the criteria for diverse MRs has posed a challenge. Traditional metrics like Mutation Score (MS) and Fault Detection Rate (FDR) fail to assess the diversity of MRs. This paper introduces the MUT Model as a foundational framework for analyzing the "MR Diversity" between a pair of MRs. The word "diversity" in this paper pertains to the differences in the types of faults that two MRs are inclined to detect. The experimental findings indicate that by harnessing posterior knowledge, specifically by analyzing the MUT Model, it is possible to derive prior knowledge that can aid in the construction of Metamorphic Relations. Most importantly, the MUT Model may draw conclusions that violate intuition, exposing more details of the core essence of MR Diversity. Moreover, the concept of MR Diversity serves as a basis for MR selection, resulting in enhanced fault detection capabilities compared to the conventional MS-based approach. Additionally, it offers valuable insights into the construction of composite metamorphic relations, with the goal of amplifying their fault detection abilities beyond those of their individual MR components.