2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)最新文献

Evaluating a Test Automation Decision Support Tool 评估测试自动化决策支持工具

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00034

Kesina Baral, Rasika Mohod, Jennifer Flamm, S. Goldrich, P. Ammann

{"title":"Evaluating a Test Automation Decision Support Tool","authors":"Kesina Baral, Rasika Mohod, Jennifer Flamm, S. Goldrich, P. Ammann","doi":"10.1109/ICSTW.2019.00034","DOIUrl":"https://doi.org/10.1109/ICSTW.2019.00034","url":null,"abstract":"Goldrich and Flamm developed the MITRE Automated Test Decision Framework (ATDF) to help MITRE government sponsors (and, via sharing on GitHub, development organizations in general) move from manually tested legacy software towards automated test, continuous integration, continuous deployment, and, ultimately, DevOps. Often such legacy systems comprise multiple components with manual test procedures. The objective of the empirical study described in this paper is to determine whether ATDF usefully ranks components with respect to Return on Investment (ROI) when introducing automated tests. ROI is simply the ratio of profit to cost. When adding automated tests, what will be the profit that these tests will carry? What is the cost or level of effort to engineer a sufficient set of automated tests? Our evaluation approach models ROI using static defect counts identified by SonarLint and estimated cost to complete testing. We found positive Pearson correlations between normalized ATDF rankings versus the normalized rankings of our evaluation approach. We reject the null hypothesis that there is no correlation between the two rankings.","PeriodicalId":310230,"journal":{"name":"2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124770788","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Practical Combinatorial Testing for XSS Detection using Locally Optimized Attack Models 基于局部优化攻击模型的跨站攻击检测组合测试

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00040

D. Simos, Bernhard Garn, Jovan Zivanovic, Manuel Leithner

{"title":"Practical Combinatorial Testing for XSS Detection using Locally Optimized Attack Models","authors":"D. Simos, Bernhard Garn, Jovan Zivanovic, Manuel Leithner","doi":"10.1109/ICSTW.2019.00040","DOIUrl":"https://doi.org/10.1109/ICSTW.2019.00040","url":null,"abstract":"In this paper, we present a combinatorial testing methodology for automated black-box security testing of complex web applications. The focus of our work is the identification of Cross-site Scripting (XSS) vulnerabilities. We introduce a new modelling scheme for test case generation of XSS attack vectors consisting of locally optimized attack models. The modelling approach takes into account the response and behavior of the web application and is particularly efficient when used in conjunction with combinatorial testing. In addition to the modelling scheme, we present a research prototype of a security testing tool called XSSInjector, which executes attack vectors generated from our methodology against web applications. The tool also employs a newly developed test oracle for detecting XSS which allow us to precisely identify whether injected JavaScript is actually executed and thus eliminate false positives. Our testing methodology is sufficiently generic to be applied to any web application that returns HTML code. We describe the foundations of our approach and validate it via an extensive case study using a verification framework and real world web applications. In particular, we have found several new critical vulnerabilities in popular forum software, library management systems and gallery packages.","PeriodicalId":310230,"journal":{"name":"2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116254120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Estimating the Number of Equivalent Mutants 估计等效突变体的数量

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00039

Amani Ayad, Imen Marsit, JiMeng Loh, Mohamed Nazih Omri, A. Mili

引用次数: 3

Developing a QRNG ECU for Automotive Security: Experience of Testing in the Real-World 开发用于汽车安全的 QRNG ECU：实际测试经验

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00033

N. H. Nga, S. Tavakoli, S. Shaikh, Oliver Maynard

引用次数: 7

A Model-Based Approach to Generate Dynamic Synthetic Test Data: A Conceptual Model 基于模型的动态综合测试数据生成方法:一个概念模型

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00026

Chao Tan, Razieh Behjati, E. Arisholm

引用次数: 3

Measuring Combinatorial Coverage at Adobe 测量Adobe的组合覆盖率

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00052

Riley Smith, Darryl C. Jarman, Jared Bellows, D. R. Kuhn, R. Kacker, D. Simos

{"title":"Measuring Combinatorial Coverage at Adobe","authors":"Riley Smith, Darryl C. Jarman, Jared Bellows, D. R. Kuhn, R. Kacker, D. Simos","doi":"10.1109/ICSTW.2019.00052","DOIUrl":"https://doi.org/10.1109/ICSTW.2019.00052","url":null,"abstract":"Adobe offers an analytics product as part of the Marketing Cloud software with which customers can track many details about users across various digital platforms. For the most part, customers define the amount and type of data to track. In addition, customers can specify many feature combinations when reporting on this data. These features create high dimensionality that makes validation challenging for some of the most critical components of the Adobe Analytics product. One of these critical components is the reporting engine. This component has a validation framework often qualitatively considered within the engineering organization as highly effective. However, the effectiveness of this framework has never been quantitatively measured. Due to recent applications of combinatorial testing, the Analytics Tools team determined to use combinatorial coverage measurements (CCM) to evaluate the effectiveness of the Replay validation framework. In this paper, we therefore report the practical application of combinatorial coverage measurements to evaluate the effectiveness of the validation framework for the Adobe Analytics reporting engine. The results of this evaluation show that combinatorial coverage measurements are an effective way to supplement existing validation for several purposes. In addition, we report details of the approach used to parse moderately nested data for use with the combinatorial coverage measurement tools.","PeriodicalId":310230,"journal":{"name":"2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132517493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Applying Combinatorial Testing to Large-Scale Data Processing at Adobe 将组合测试应用于Adobe的大规模数据处理

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00051

Riley Smith, Darryl C. Jarman, R. Kacker, D. R. Kuhn, D. Simos, Ludwig Kampel, Manuel Leithner, Gabe Gosney

{"title":"Applying Combinatorial Testing to Large-Scale Data Processing at Adobe","authors":"Riley Smith, Darryl C. Jarman, R. Kacker, D. R. Kuhn, D. Simos, Ludwig Kampel, Manuel Leithner, Gabe Gosney","doi":"10.1109/ICSTW.2019.00051","DOIUrl":"https://doi.org/10.1109/ICSTW.2019.00051","url":null,"abstract":"Adobe offers an analytics product as part of the Marketing Cloud software with which customers can track many details about users across various digital platforms. For the most part, customers define the amount and type of data to track. This high dimensionality makes validation difficult or intractable. Due to increasing attention from both industry and academia, combinatorial testing was investigated and applied to improve existing validation. In this paper, we report the practical application of combinatorial testing to the data collection, compression and processing components of the Adobe analytics product. Consequently, the effectiveness of combinatorial testing for this application is measured in terms of new defects found rather than detecting known defects from previous versions. The results of the application show that combinatorial testing is an effective way to improve validation for these components of Adobe Analytics. In addition, we report the details of the input parameter modeling process and test value selection to provide more context for the problem and how combinatorial testing provides the structure to improve validation for Adobe Analytics.","PeriodicalId":310230,"journal":{"name":"2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114036047","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 14

Variable Strength Combinatorial Testing for Deep Neural Networks 深度神经网络的变强度组合测试

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00066

Yanshan Chen, Ziyuan Wang, Dong Wang, Chunrong Fang, Zhenyu Chen

引用次数: 6

AbsCon: A Test Concretizer for Model-Based Testing 基于模型测试的测试具体器

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00027

J. Vanhecke, Xavier Devroey, Gilles Perrouin

{"title":"AbsCon: A Test Concretizer for Model-Based Testing","authors":"J. Vanhecke, Xavier Devroey, Gilles Perrouin","doi":"10.1109/ICSTW.2019.00027","DOIUrl":"https://doi.org/10.1109/ICSTW.2019.00027","url":null,"abstract":"Test definition and execution is an essential but time-consuming task during system development. To speed up the process, model-based testing and other related approaches propose to generate/write abstract test cases and to concretize them using either transformations, an adapter, or a mixture of the two. QTaste is an industrial data-driven test case definition and execution environment used to perform black-box testing on various kinds of systems. QTaste's test cases are manually written in Python and use an adapter, called test API, to execute operations on the System Under Test (SUT) interfaces. In this paper, we describe AbsCon (Abstract test case Concretizer), a plugin used to generate test cases executable in QTaste based on their definition: i.e., sequences of abstract actions and assertions. AbsCon uses programmer friendly mappings (written in Python) for the SUT's interfaces, actions, and assertions, to generate standard test cases in QTaste format. Rather than having a complete model-based testing transformation chain, the plugin is bridging the gap between existing model-based test case generation tools and an industrial test case execution system using a mix of transformation and adaptation.","PeriodicalId":310230,"journal":{"name":"2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"121 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123600217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Weighted Combinatorial Sequence Testing for the TLS Protocol TLS协议的加权组合序列测试

2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) Pub Date : 2019-04-22 DOI: 10.1109/ICSTW.2019.00031

Bernhard Garn, D. Simos, Feng Duan, Yu Lei, Josip Bozic, F. Wotawa

引用次数: 5