A Comprehensive Framework for Testing Goal-Oriented NFPs in Software Product Lines

Abstract

In the realm of software product line engineering (SPLE), ensuring the quality of end products is paramount for market success. SPLE promotes systematic software development through reuse by focusing on commonalities and variabilities within a domain to efficiently produce a family of related systems. The quality of a software system depends on its functional properties (FPs)—the functionalities it provides—and its non-functional properties (NFPs)—the quality attributes it possesses, such as security and performance. NFPs are particularly critical because they directly impact user satisfaction, determine project success, and significantly influence market acceptance. However, in SPLE, despite their recognized importance, NFPs often receive less attention compared to FPs, leading to potential quality risks and increased costs. This paper presents a framework for testing goal-oriented NFPs in software product lines, addressing this gap. By integrating goal models, the framework supports the systematic capture and validation of NFPs from early development stages. The framework's applicability is illustrated through research-based case studies in an online bookstore product line, demonstrating its use for systematic NFPs testing at both the domain and application levels. A comparative analysis with an existing technique highlights the framework's unique contributions in addressing NFPs testing within software product lines. Additionally, a preliminary experiment using two widely recognized product line domain examples evaluated the core testing process supported by the framework during the domain engineering phase, focusing on effectiveness, performance efficiency, and time consistency in structured research settings.