How (Much) Do Developers Test?

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering Pub Date : 2015-05-16 DOI:10.1109/ICSE.2015.193

M. Beller, Georgios Gousios, A. Zaidman

{"title":"How (Much) Do Developers Test?","authors":"M. Beller, Georgios Gousios, A. Zaidman","doi":"10.1109/ICSE.2015.193","DOIUrl":null,"url":null,"abstract":"What do we know about software testing in the real world? It seems we know from Fred Brooks' seminal work \"The Mythical Man-Month\" that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past 40 years, the question stands: Is this observation still true? In fact, was it ever true? The vision for our research is to settle the discussion about Brooks' estimation once and for all: How much do developers test? Does developers' estimation on how much they test match reality? How frequently do they execute their tests, and is there a relationship between test runtime and execution frequency? What are the typical reactions to failing tests? Do developers solve actual defects in the production code, or do they merely relax their test assertions? Emerging results from 40 software engineering students show that students overestimate their testing time threefold, and 50% of them test as little as 4% of their time, or less. Having proven the scalability of our infrastructure, we are now extending our case study with professional software engineers from open-source and industrial organizations.","PeriodicalId":330487,"journal":{"name":"2015 IEEE/ACM 37th IEEE International Conference on Software Engineering","volume":"13 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"55","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2015 IEEE/ACM 37th IEEE International Conference on Software Engineering","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICSE.2015.193","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 55

Abstract

What do we know about software testing in the real world? It seems we know from Fred Brooks' seminal work "The Mythical Man-Month" that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past 40 years, the question stands: Is this observation still true? In fact, was it ever true? The vision for our research is to settle the discussion about Brooks' estimation once and for all: How much do developers test? Does developers' estimation on how much they test match reality? How frequently do they execute their tests, and is there a relationship between test runtime and execution frequency? What are the typical reactions to failing tests? Do developers solve actual defects in the production code, or do they merely relax their test assertions? Emerging results from 40 software engineering students show that students overestimate their testing time threefold, and 50% of them test as little as 4% of their time, or less. Having proven the scalability of our infrastructure, we are now extending our case study with professional software engineers from open-source and industrial organizations.

查看原文本刊更多论文

开发人员测试多少?

我们对现实世界中的软件测试了解多少?我们似乎从Fred Brooks的开创性著作“神话般的人月”中知道，50%的项目工作都花在了测试上。然而，由于软件工程在过去的40年里取得了巨大的进步，问题出现了:这个观察结果仍然正确吗?事实上，这是真的吗?我们研究的愿景是一劳永逸地解决关于Brooks估计的讨论:开发人员测试了多少?开发人员对测试量的估计与现实相符吗?他们执行测试的频率是多少，测试运行时和执行频率之间是否存在关系?测试失败的典型反应是什么?开发人员是解决了产品代码中的实际缺陷，还是仅仅放松了他们的测试断言?来自40名软件工程专业学生的最新结果表明，学生们将他们的测试时间高估了三倍，其中50%的人只测试了他们时间的4%，或者更少。在证明了我们的基础设施的可伸缩性之后，我们现在正在与来自开源和工业组织的专业软件工程师一起扩展我们的案例研究。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

自引率

0.00%

发文量