在大型工业软件项目中，覆盖率对Bug密度的影响

2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) Pub Date : 2017-11-09 DOI:10.1109/ESEM.2017.44

Thomas Bach, A. Andrzejak, Ralf Pannemans, D. Lo

{"title":"在大型工业软件项目中，覆盖率对Bug密度的影响","authors":"Thomas Bach, A. Andrzejak, Ralf Pannemans, D. Lo","doi":"10.1109/ESEM.2017.44","DOIUrl":null,"url":null,"abstract":"Measuring quality of test suites is one of the major challenges of software testing. Code coverage identifies tested and untested parts of code and is frequently used to approximate test suite quality. Multiple previous studies have investigated the relationship between coverage ratio and test suite quality, without a clear consent in the results. In this work we study whether covered code contains a smaller number of future bugs than uncovered code (assuming appropriate scaling). If this correlation holds and bug density is lower in covered code, coverage can be regarded as a meaningful metric to estimate the adequacy of testing. To this end we analyse 16000 internal bug reports and bug-fixes of SAP HANA, a large industrial software project. We found that the above-mentioned relationship indeed holds, and is statistically significant. Contrary to most previous works our study uses real bugs and real bug-fixes. Furthermore, our data is derived from a complex and large industrial project.","PeriodicalId":213866,"journal":{"name":"2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2017-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"17","resultStr":"{\"title\":\"The Impact of Coverage on Bug Density in a Large Industrial Software Project\",\"authors\":\"Thomas Bach, A. Andrzejak, Ralf Pannemans, D. Lo\",\"doi\":\"10.1109/ESEM.2017.44\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Measuring quality of test suites is one of the major challenges of software testing. Code coverage identifies tested and untested parts of code and is frequently used to approximate test suite quality. Multiple previous studies have investigated the relationship between coverage ratio and test suite quality, without a clear consent in the results. In this work we study whether covered code contains a smaller number of future bugs than uncovered code (assuming appropriate scaling). If this correlation holds and bug density is lower in covered code, coverage can be regarded as a meaningful metric to estimate the adequacy of testing. To this end we analyse 16000 internal bug reports and bug-fixes of SAP HANA, a large industrial software project. We found that the above-mentioned relationship indeed holds, and is statistically significant. Contrary to most previous works our study uses real bugs and real bug-fixes. Furthermore, our data is derived from a complex and large industrial project.\",\"PeriodicalId\":213866,\"journal\":{\"name\":\"2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)\",\"volume\":\"5 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2017-11-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"17\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ESEM.2017.44\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ESEM.2017.44","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 17

摘要

测量测试套件的质量是软件测试的主要挑战之一。代码覆盖率标识代码的已测试和未测试部分，并且经常用于近似测试套件的质量。许多先前的研究已经调查了覆盖率和测试套件质量之间的关系，但在结果中没有明确的一致意见。在这项工作中，我们研究被覆盖的代码是否比未覆盖的代码包含更少的未来bug(假设适当的缩放)。如果这种相关性成立，并且在被覆盖的代码中bug密度较低，覆盖率可以被视为评估测试充分性的有意义的度量。为此，我们分析了SAP HANA(一个大型工业软件项目)的16000个内部bug报告和bug修复。我们发现上述关系确实成立，并且具有统计学意义。与之前的大多数工作相反，我们的研究使用了真正的bug和真正的bug修复。此外，我们的数据来源于一个复杂的大型工业项目。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

The Impact of Coverage on Bug Density in a Large Industrial Software Project

Measuring quality of test suites is one of the major challenges of software testing. Code coverage identifies tested and untested parts of code and is frequently used to approximate test suite quality. Multiple previous studies have investigated the relationship between coverage ratio and test suite quality, without a clear consent in the results. In this work we study whether covered code contains a smaller number of future bugs than uncovered code (assuming appropriate scaling). If this correlation holds and bug density is lower in covered code, coverage can be regarded as a meaningful metric to estimate the adequacy of testing. To this end we analyse 16000 internal bug reports and bug-fixes of SAP HANA, a large industrial software project. We found that the above-mentioned relationship indeed holds, and is statistically significant. Contrary to most previous works our study uses real bugs and real bug-fixes. Furthermore, our data is derived from a complex and large industrial project.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

自引率

0.00%

发文量