A Two-staged Survey on Release Readiness

Abstract

Deciding about the content and readiness when shipping a new product release can have a strong impact on the success (or failure) of the product. Having formerly analyzed the state-of-the art in this area, the objective for this paper was to better understand the process and rationale of real-world release decisions and to what extent research on release readiness is aligned with industrial needs. We designed two rounds of surveys with focus on the current (Survey-A) and the desired (Survey-B) process of how to make release readiness decisions. We received 49 and 40 valid responses for Survey-A and Survey-B, respectively. In total, we identified 12 main findings related to the process, the rationale and the tool support considered for making release readiness decisions. We found that reasons for failed releases and the factors considered for making release decisions are context specific and vary with release cycle time. Practitioners confirmed that (i) release readiness should be measured and continuously monitored during the whole release cycle, (ii) release readiness decisions are context-specific and should not be based solely on quality considerations, and iii) some of the observed reasons for failed releases such as low functionality, high cost, and immature service are not adequately studied in research where there is dominance on investigating quality and testing only. In terms of requested tool support, dashboards covering multidimensional aspects of the status of release development were articulated as key requirements.