Governance and Cost Reduction through Multi-tier Preventive Performance Tests in a Large-Scale Product Line Development

Abstract

Experience has shown that maintaining software system performance in a complex product line development is a constant challenge, already achieved performance is often degraded over time because proper quality gates are rarely defined or implemented. The established practice of performance verification tests on an integrated software baseline is essential to ensure final quality of the delivered products, but is late if performance degradations already crept in. Maintenance of performance in software baselines requires an additional preventive approach. The faulty software changes that degrade performance can be identified (performance quality gates) before these changes can flow into the baseline and subsequently get rejected. This ensures that the software baseline maintains a consistent performance leading to more predictable schedules and development costs. For a complex software family involving parallel and dependent sub-projects of domain platforms and end user applications, these performance quality gates need to be established at multiple levels.