Model-based tests of truisms

Abstract

Software engineering (SE) truisms capture broadly-applicable principles of software construction. The trouble with truisms is that such general principles may not apply in specific cases. This paper tests the specificity of two SE truisms: (a) increasing software process level is a desirable goal; and (b) it is best to remove errors during the early parts of a software lifecycle. Our tests are based on two well-established SE models: (1) Boehm et.al.'s COCOMO II cost estimation model; and (2) Raffo's discrete event software process model of a software project life cycle. After extensive simulations of these models, the TAR2 treatment learner was applied to find the model parameters that most improved the potential performance of the real-world systems being modelled. The case studies presented here showed that these truisms are clearly sub-optimal for certain projects since other factors proved to be far more critical. Hence, we advise against truism-based process improvement. This paper offers a general alternative framework for model-based assessment of methods to improve software quality: modelling + validation + simulation + sensitivity. That is, after recording what is known in a model, that model should be validated, explored using simulations, then summarized to find the key factors that most improve model behavior.