Characterizing safe and partially safe evolution scenarios in product lines: An Empirical Study

Evolving software product lines is often error-prone. Previous works have proposed classifying product line evolution into safe or partially safe, depending on the number of products that have their behavior preserved after evolution. Based on these notions, it is possible to derive transformation templates that abstract common evolution scenarios, such as adding an optional feature. However, existing works are focused on evaluating either safe or partially safe templates. Hence, in this work we aim to characterize product line evolution as a whole, measuring to what extent the evolution history is safe compared to partially safe, to better understand how product lines evolve. We measure how often existing templates happen using 2,300 commits from an open-source product line. According to our study, 91.7% of the commits represent partially safe evolution scenarios. Our results also show that 1,800 of these commits can automatically be classified as instances of existing templates. Among these, commits that do not modify other variability-aware models, are the most frequent, accounting for 72.3% out of the total of commits. For the remaining 500 commits, we identify that 24.4% are related to changes in the configuration knowledge, that is, the file responsible for the mapping between features and code.