Monitoring bottlenecks in achieving release readiness: a retrospective case study across ten OSS projects

Context: Not releasing software on time can cause substantial loss in revenue. Continuous awareness of the product release status is required. Release readiness is a time-dependent attribute of the status of the product release, which aggregates the degree of satisfaction of a portfolio of release process and product measures. Goal: At different stages of a release cycle, the goal is to understand frequencies and pattern of occurrence of factors affecting project success by restricting the status of release readiness (called bottlenecks). Method: As a form of explorative case study research, we analyzed ten open source software (OSS) projects taken from the GitHub repository. As a retrospective study covering a period of 28 weeks, we monitored eight release readiness attributes and identified their impact on release readiness over time across the ten projects. Results: Feature completion rate, Bug fixing rate, and Features implemented were observed as the most frequent bottlenecks. The most frequent transition between bottlenecks is from Pull-request completion rate to Bug fixing rate. With the exception of Pull-request completion rate, no significant differences were found in occurrence of bottleneck factors between early and late stage of the release cycle. Conclusions: We received an initial understanding of the most frequent bottleneck factors for release readiness and their likelihood of subsequent occurrence. This is intended to guide the effort spent on improving release engineering.