A Study on the Impact of Model Evolution in MBT Suites

Abstract

Software testing is known to be a key-factor for the success of a software development project. In this context, Model-Based Testing (MBT) plays an important role by providing an automated way for generating test cases from system models. However, although MBT can be helpful for creating sound test cases, its use can be very sensitive to model changes. Model edits often lead to a great number of obsolete test cases, as the software and its models evolve. This fact is even more evident in agile projects where requirement artifacts are very volatile. This paper presents an empirical study designed for investigating how model edits can impact MBT test suites. For that, we run a case study in the context of two industrial projects that apply agile methodologies combined with the use of MBT. We observed the evolution of specification models and their impact on generated MBT suites. Our results showed that 86% of the generated test cases were discarded due to model edits and their impact. However, a deeper analysis found that 52% of these tests were impacted only by syntactic model edits, which indicate they could be reused with little revision efforts.