A Large-Scale Technology Evaluation Study: Effects of Model-based Analysis and Testing

Abstract

Besides model-based development, model-based quality assurance and the tighter integration of static and dynamic quality assurance activities are becoming increasingly relevant in the development of software-intensive systems. Thus, this paper reports on an empirical study aimed at investigating the promises regarding quality improvements and cost savings. The evaluation comprises data from 13 industry case studies conducted during a three-year large-scale research project in the transportation domain (automotive, avionics, rail system). During the evaluation, we identified major goals and strategies associated with (integrated) model-based analysis and testing and evaluated the improvements achieved. The aggregated results indicate an average cost reduction of between 29% and 34% for verification and validation and of between 22% and 32% for defect removal. Compared with these cost savings, improvements regarding test coverage (~8%), number of remaining defects (~13%), and time to market (~8%) appear less noticeable.