Applicability of Software Reliability Growth Models to Open Source Software

Software Reliability Growth Models (SRGMs) are based on underlying assumptions which make them typically more suited for quality evaluation of closed-source projects and their development lifecycles. Their usage in open-source software (OSS) projects is a subject of debate. Although the studies investigating the SRGMs applicability in OSS context do exist, they are limited by the number of models and projects considered which might lead to inconclusive results. In this paper, we present an experimental study of SRGMs applicability to a total of 88 OSS projects, comparing nine SRGMs, looking at the stability of the best models on the whole projects, on releases, on different domains, and according to different projects’ attributes. With the aid of the STRAIT tool, we automated repository mining, data processing, and SRGM analysis for better reproducibility. Overall, we found good applicability of SRGMs to OSS, but with different performance when segmenting the dataset into releases and domains, highlighting the difficulty in generalizing the findings and in the search for one-fits-all models.