2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)最新文献

Empirical Evaluation of Test Coverage for Functional Programs 功能程序测试覆盖率的实证评估

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.8

Yufeng Cheng, Meng Wang, Yingfei Xiong, Dan Hao, Lu Zhang

{"title":"Empirical Evaluation of Test Coverage for Functional Programs","authors":"Yufeng Cheng, Meng Wang, Yingfei Xiong, Dan Hao, Lu Zhang","doi":"10.1109/ICST.2016.8","DOIUrl":"https://doi.org/10.1109/ICST.2016.8","url":null,"abstract":"The correlation between test coverage and test effectiveness is important to justify the use of coverage in practice. Existing results on imperative programs mostly show that test coverage predicates effectiveness. However, since functional programs are usually structurally different from imperative ones, it is unclear whether the same result may be derived and coverage can be used as a prediction of effectiveness on functional programs. In this paper we report the first empirical study on the correlation between test coverage and test effectiveness on functional programs. We consider four types of coverage: as input coverages, statement/branch coverage and expression coverage, and as oracle coverages, count of assertions and checked coverage. We also consider two types of effectiveness: raw effectiveness and normalized effectiveness. Our results are twofold. (1) In general the findings on imperative programs still hold on functional programs, warranting the use of coverage in practice. (2) On specific coverage criteria, the results may be unexpected or different from the imperative ones, calling for further studies on functional programs.","PeriodicalId":155554,"journal":{"name":"2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124166254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Generating Evil Test Strings for Regular Expressions 为正则表达式生成错误的测试字符串

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.29

E. Larson, Anna Kirk

引用次数: 24

Why do Record/Replay Tests of Web Applications Break? 为什么Web应用程序的记录/重播测试会中断?

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.16

Mouna Hammoudi, G. Rothermel, P. Tonella

引用次数: 41

A Controlled Experiment in Testing of Safety-Critical Embedded Software 安全关键型嵌入式软件测试的控制实验

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.15

Eduard Paul Enoiu, Adnan Causevic, Daniel Sundmark, P. Pettersson

{"title":"A Controlled Experiment in Testing of Safety-Critical Embedded Software","authors":"Eduard Paul Enoiu, Adnan Causevic, Daniel Sundmark, P. Pettersson","doi":"10.1109/ICST.2016.15","DOIUrl":"https://doi.org/10.1109/ICST.2016.15","url":null,"abstract":"In engineering of safety critical systems, regulatory standards often put requirements on both traceable specification-based testing, and structural coverage on program units. Automated test generation techniques can be used to generate inputs to cover the structural aspects of a program. However, there is no conclusive evidence on how automated test generation compares to manual test design, or how testing based on the program implementation relates to specification-based testing. In this paper, we investigate specification -- and implementation-based testing of embedded software written in the IEC 61131-3 language, a programming standard used in many embedded safety critical software systems. Further, we measure the efficiency and effectiveness in terms of fault detection. For this purpose, a controlled experiment was conducted, comparing tests created by a total of twenty-three software engineering master students. The participants worked individually on manually designing and automatically generating tests for two IEC 61131-3 programs. Tests created by the participants in the experiment were collected and analyzed in terms of mutation score, decision coverage, number of tests, and testing duration. We found that, when compared to implementation-based testing, specification-based testing yields significantly more effective tests in terms of the number of faults detected. Specifically, specification-based tests more effectively detect comparison and value replacement type of faults, compared to implementation-based tests. On the other hand, implementation-based automated test generation leads to fewer tests (up to 85% improvement) created in shorter time than the ones manually created based on the specification.","PeriodicalId":155554,"journal":{"name":"2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115087153","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 18

Using Visual Symptoms for Debugging Presentation Failures in Web Applications 使用可视化症状调试Web应用程序中的表示失败

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.35

Sonal Mahajan, Bailan Li, Pooyan Behnamghader, William G. J. Halfond

引用次数: 40

Large Scale Generation of Complex and Faulty PHP Test Cases 大规模生成复杂和错误的PHP测试用例

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.43

Bertrand Stivalet, E. Fong

引用次数: 20

Predicting Testability of Concurrent Programs 预测并发程序的可测试性

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.39

Tingting Yu, Wei Wen, Xue Han, J. Hayes

{"title":"Predicting Testability of Concurrent Programs","authors":"Tingting Yu, Wei Wen, Xue Han, J. Hayes","doi":"10.1109/ICST.2016.39","DOIUrl":"https://doi.org/10.1109/ICST.2016.39","url":null,"abstract":"Concurrent programs are difficult to test due to their inherent non-determinism. To address the nondeterminism problem, testing often requires the exploration of thread schedules of a program, this can be time-consuming for testing real-world programs. We believe that testing resources can be distributed more effectively if testability of concurrent programs can be estimated, so that developers can focus on exploring the low testable code. Voas introduces a notion of testability as the probability that a test case will fail if the program has a fault, in which testability can be measured based on fault-based testing and mutation analysis. Much research has been proposed to analyze testability and predict defects for sequential programs, but to date, no work has considered testability prediction for concurrent programs, with program characteristics distinguished from sequential programs. In this paper, we present an approach to predict testability of concurrent programs at the function level. We propose a set of novel static code metrics based on the unique properties of concurrent programs. To evaluate the performance of our approach, we build a family of testability prediction models combining both static metrics and a test suite metric and apply it to real projects. Our empirical study reveals that our approach is more accurate than existing sequential program metrics.","PeriodicalId":155554,"journal":{"name":"2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123282915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

How Well are Your Requirements Tested? 你的需求测试得有多好?

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.23

T. Arts, John Hughes

引用次数: 4

Selecting the Right Topics for Industry-Academia Collaborations in Software Testing: An Experience Report 为软件测试中的产学研合作选择合适的主题:一份经验报告

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.7

V. Garousi, Kadir Herkiloglu

{"title":"Selecting the Right Topics for Industry-Academia Collaborations in Software Testing: An Experience Report","authors":"V. Garousi, Kadir Herkiloglu","doi":"10.1109/ICST.2016.7","DOIUrl":"https://doi.org/10.1109/ICST.2016.7","url":null,"abstract":"The global software industry and the Software Engineering (SE) academia are two large communities. However, unfortunately, the level of joint industry-academia collaborations (IAC) in SE is still relatively very low, compared to the amount of activity in each of the two communities. Selecting the right topic for a new IAC has been reported to be challenging and often a deal-maker or-breaker for the start of IACs. Motivated by the above need, the goal of this paper is to propose experience-based guidelines from our 10+ software testing IACs in the past several years in Canada and Turkey to effectively and efficiently select right topics for IACs in software testing (also easily generalizable to other areas of SE), for the benefit of SE researchers and practitioners in starting new IACs. The experience and evidence supporting the guidelines in this paper are drawn from the authors' past projects and also seven on-going software-testing projects in the context of a large Turkish software and systems company. The topic-selection process has involved interaction with company representatives in the form of both multiple group discussions and separate face-to-face meetings while utilizing grounded-theory to find (converge to) topics which would be 'interesting' and useful from both industrial and academic perspectives. To increase the success of our topic selection process, we also utilized two other sources of information from the literature: (1) a set of four fitness criteria for topic selection in industry experiments, and (2) challenges and best practices for IAC, specific to project inception, as synthesized in a recent systematic literature review. We believe the results of this paper would be helpful for other researchers and practitioners not only in software testing but also in software engineering in general in increasing their chances of success in project inception and topic selection phase.","PeriodicalId":155554,"journal":{"name":"2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123058671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 21

Automatically Discovering, Reporting and Reproducing Android Application Crashes 自动发现，报告和再现Android应用程序崩溃

2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2016-04-11 DOI: 10.1109/ICST.2016.34

Kevin Moran, M. Vásquez, Carlos Bernal-Cárdenas, Christopher Vendome, D. Poshyvanyk

{"title":"Automatically Discovering, Reporting and Reproducing Android Application Crashes","authors":"Kevin Moran, M. Vásquez, Carlos Bernal-Cárdenas, Christopher Vendome, D. Poshyvanyk","doi":"10.1109/ICST.2016.34","DOIUrl":"https://doi.org/10.1109/ICST.2016.34","url":null,"abstract":"Mobile developers face unique challenges when detecting and reporting crashes in apps due to their prevailing GUI event-driven nature and additional sources of inputs (e.g., sensor readings). To support developers in these tasks, we introduce a novel, automated approach called CRASHSCOPE. This tool explores a given Android app using systematic input generation, according to several strategies informed by static and dynamic analyses, with the intrinsic goal of triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented crash report containing screenshots, detailed crash reproduction steps, the captured exception stack trace, and a fully replayable script that automatically reproduces the crash on a target device(s). We evaluated CRASHSCOPE's effectiveness in discovering crashes as compared to five state-of-the-art Android input generation tools on 61 applications. The results demonstrate that CRASHSCOPE performs about as well as current tools for detecting crashes and provides more detailed fault information. Additionally, in a study analyzing eight real-world Android app crashes, we found that CRASHSCOPE's reports are easily readable and allow for reliable reproduction of crashes by presenting more explicit information than human written reports.","PeriodicalId":155554,"journal":{"name":"2016 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124115183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 160