The Impact of Continuous Code Quality Assessment on Defects

Abstract

Continuous Code Quality Assessment (CCQA) tools promise that increasing code quality leads to fewer defects, i.e., that software quality from the user view can be increased by increasing quality of the product. Currently, there is limited evidence on that application of CCQA tools, such as, SonarCloud (SC), during software development actually reduces the amount of defects over time. In this paper we study five open-source projects that adopt SonarCloud (SC) for CCQA and we compare frequencies of defect reports before and after adoption of SC. For only one project (Apache Ratis), we find a statistically significant decrease of defects after adoption of the tool. After closer investigation we find, that this decrease is likely just a coincidence and not caused by the adoption of SC and adherence to its code quality recommendations. In general, we find no evidence for that application of a CCQA tool increases product quality.