2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)最新文献_第2页

Automata Language Equivalence vs. Simulations for Model-Based Mutant Equivalence: An Empirical Evaluation 自动机语言等价与基于模型的突变等价的模拟:一个经验评价

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.46

Xavier Devroey, Gilles Perrouin, Mike Papadakis, Axel Legay, Pierre-Yves Schobbens, P. Heymans

{"title":"Automata Language Equivalence vs. Simulations for Model-Based Mutant Equivalence: An Empirical Evaluation","authors":"Xavier Devroey, Gilles Perrouin, Mike Papadakis, Axel Legay, Pierre-Yves Schobbens, P. Heymans","doi":"10.1109/ICST.2017.46","DOIUrl":"https://doi.org/10.1109/ICST.2017.46","url":null,"abstract":"Mutation analysis is a popular test assessment method. It relies on the mutation score, which indicates how many mutants are revealed by a test suite. Yet, there are mutants whose behaviour is equivalent to the original system, wasting analysis resources and preventing the satisfaction of the full (100%) mutation score. For finite behavioural models, the Equivalent Mutant Problem (EMP) can be addressed through language equivalence of non-deterministic finite automata, which is a well-studied, yet computationally expensive, problem in automata theory. In this paper, we report on our preliminary assessment of a state-of-the-art exact language equivalence tool to handle the EMP against 3 models of size up to 15,000 states on 1170 mutants. We introduce random and mutation-biased simulation heuristics as baselines for comparison. Results show that the exact approach is often more than ten times faster in the weak mutation scenario. For strong mutation, our biased simulations are faster for models larger than 300 states. They can be up to 1,000 times faster while limiting the error of misclassifying non-equivalent mutants as equivalent to 10% on average. We therefore conclude that the approaches can be combined for improved efficiency.","PeriodicalId":112258,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116105977","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Recovering Semantic Traceability Links between APIs and Security Vulnerabilities: An Ontological Modeling Approach 恢复api和安全漏洞之间的语义可追溯性链接:一种本体建模方法

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.15

Sultan S. Al-Qahtani, Ellis E. Eghan, J. Rilling

{"title":"Recovering Semantic Traceability Links between APIs and Security Vulnerabilities: An Ontological Modeling Approach","authors":"Sultan S. Al-Qahtani, Ellis E. Eghan, J. Rilling","doi":"10.1109/ICST.2017.15","DOIUrl":"https://doi.org/10.1109/ICST.2017.15","url":null,"abstract":"Over the last decade, a globalization of the software industry took place, which facilitated the sharing and reuse of code across existing project boundaries. At the same time, such global reuse also introduces new challenges to the software engineering community, with not only components but also their problems and vulnerabilities being now shared. For example, vulnerabilities found in APIs no longer affect only individual projects but instead might spread across projects and even global software ecosystem borders. Tracing these vulnerabilities at a global scale becomes an inherently difficult task since many of the existing resources required for such analysis still rely on proprietary knowledge representation. In this research, we introduce an ontology-based knowledge modeling approach that can eliminate such information silos. More specifically, we focus on linking security knowledge with other software knowledge to improve traceability and trust in software products (APIs). Our approach takes advantage of the Semantic Web and its reasoning services, to trace and assess the impact of security vulnerabilities across project boundaries. We present a case study, to illustrate the applicability and flexibility of our ontological modeling approach by tracing vulnerabilities across project and resource boundaries.","PeriodicalId":112258,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123669907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 17

O!Snap: Cost-Efficient Testing in the Cloud O !Snap:云中的成本效益测试

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.51

Alessio Gambi, Alessandra Gorla, A. Zeller

引用次数: 14

Perphecy: Performance Regression Test Selection Made Simple but Effective 假设:性能回归测试选择简单而有效

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.17

Augusto Born de Oliveira, S. Fischmeister, Amer Diwan, Matthias Hauswirth, P. Sweeney

引用次数: 32

FIFA: A Kernel-Level Fault Injection Framework for ARM-Based Embedded Linux System 基于arm的嵌入式Linux系统的内核级故障注入框架

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.10

Eunji Jeong, Namgoo Lee, Jinhan Kim, Duseok Kang, S. Ha

引用次数: 8

Using Semantic Similarity in Crawling-Based Web Application Testing 语义相似度在基于爬虫的Web应用程序测试中的应用

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.20

Jun-Wei Lin, Farn Wang, Paul Chu

{"title":"Using Semantic Similarity in Crawling-Based Web Application Testing","authors":"Jun-Wei Lin, Farn Wang, Paul Chu","doi":"10.1109/ICST.2017.20","DOIUrl":"https://doi.org/10.1109/ICST.2017.20","url":null,"abstract":"To automatically test web applications, crawling-based techniques are usually adopted to mine the behavior models, explore the state spaces or detect the violated invariants of the applications. However, their broad use is limited by the required manual configurations for input value selection, GUI state comparison and clickable detection. In existing crawlers, the configurations are usually string-matching based rules looking for tags or attributes of DOM elements, and often application-specific. Moreover, in input topic identification, it can be difficult to determine which rule suggests a better match when several rules match an input field to more than one topic. This paper presents a natural-language approach based on semantic similarity to address the above issues. The proposed approach represents DOM elements as vectors in a vector space formed by the words used in the elements. The topics of encountered input fields during crawling can then be inferred by their similarities with ones in a labeled corpus. Semantic similarity can also be applied to suggest if a GUI state is newly discovered and a DOM element is clickable under an unsupervised learning paradigm. We evaluated the proposed approach in input topic identification with 100 real-world forms and GUI state comparison with real data from industry. Our evaluation shows that the proposed approach has comparable or better performance to the conventional techniques. Experiments in input topic identification also show that the accuracy of the rule-based approach can be improved by up to 22% when integrated with our approach.","PeriodicalId":112258,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"118 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133235709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

Behavioral Execution Comparison: Are Tests Representative of Field Behavior? 行为执行比较:测试是否代表现场行为?

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.36

Qianqian Wang, Yuriy Brun, A. Orso

{"title":"Behavioral Execution Comparison: Are Tests Representative of Field Behavior?","authors":"Qianqian Wang, Yuriy Brun, A. Orso","doi":"10.1109/ICST.2017.36","DOIUrl":"https://doi.org/10.1109/ICST.2017.36","url":null,"abstract":"Software testing is the most widely used approach for assessing and improving software quality, but it is inherently incomplete and may not be representative of how the software is used in the field. This paper addresses the questions of to what extent tests represent how real users use software, and how to measure behavioral differences between test and field executions. We study four real-world systems, one used by endusers and three used by other (client) software, and compare test suites written by the systems' developers to field executions using four models of behavior: statement coverage, method coverage, mutation score, and a temporal-invariant-based model we developed. We find that developer-written test suites fail to accurately represent field executions: the tests, on average, miss 6.2% of the statements and 7.7% of the methods exercised in the field, the behavior exercised only in the field kills an extra 8.6% of the mutants, finally, the tests miss 52.6% of the behavioral invariants that occur in the field. In addition, augmenting the in-house test suites with automatically-generated tests by a tool targeting high code coverage only marginally improves the tests' behavioral representativeness. These differences between field and test executions—and in particular the finer-grained and more sophisticated ones that we measured using our invariantbased model—can provide insight for developers and suggest a better method for measuring test suite quality.","PeriodicalId":112258,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122138643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 23

ADRENALIN-RV: Android Runtime Verification Using Load-Time Weaving 肾上腺素- rv: Android运行时验证使用加载时间编织

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.61

Haiyang Sun, Andrea Rosà, Omar Javed, Walter Binder

{"title":"ADRENALIN-RV: Android Runtime Verification Using Load-Time Weaving","authors":"Haiyang Sun, Andrea Rosà, Omar Javed, Walter Binder","doi":"10.1109/ICST.2017.61","DOIUrl":"https://doi.org/10.1109/ICST.2017.61","url":null,"abstract":"Android has become one of the most popular operating systems for mobile devices. As the number of applications for the Android ecosystem grows, so is their complexity, increasing the need for runtime verification on the Android platform. Unfortunately, despite the presence of several runtime verification frameworks for Java bytecode, DEX bytecode used in Android does not benefit from such a wide support. While a few runtime verification tools support applications developed for Android, such tools offer only limited bytecode coverage and may not be able to detect property violations in certain classes. In this paper, we show that ADRENALIN-RV, our new runtime verification tool for Android, overcomes this limitation. In contrast to other frameworks, ADRENALIN-RV weaves monitoring code at load time and is able to instrument all loaded classes. In addition to the default classes inside the application package (APK), ADRENALIN-RV covers both the Android class library and libraries dynamically loaded from the storage, network, or generated dynamically, which related tools cannot verify. Evaluation results demonstrate the increased code coverage of ADRENALIN-RV with respect to other runtime validation tools for Android. Thanks to ADRENALIN-RV, we were able to detect violations that cannot be detected by other tools.","PeriodicalId":112258,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131234092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

A Comparative Study of Manual and Automated Testing for Industrial Control Software 工业控制软件人工与自动化测试的比较研究

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.44

Eduard Paul Enoiu, Daniel Sundmark, Adnan Causevic, P. Pettersson

{"title":"A Comparative Study of Manual and Automated Testing for Industrial Control Software","authors":"Eduard Paul Enoiu, Daniel Sundmark, Adnan Causevic, P. Pettersson","doi":"10.1109/ICST.2017.44","DOIUrl":"https://doi.org/10.1109/ICST.2017.44","url":null,"abstract":"Automated test generation has been suggested as a way of creating tests at a lower cost. Nonetheless, it is not very well studied how such tests compare to manually written ones in terms of cost and effectiveness. This is particularly true for industrial control software, where strict requirements on both specification-based testing and code coverage typically are met with rigorous manual testing. To address this issue, we conducted a case study in which we compared manually and automatically created tests. We used recently developed real-world industrial programs written in the IEC 61131-3, a popular programming language for developing industrial control systems using programmable logic controllers. The results show that automatically generated tests achieve similar code coverage as manually created tests, but in a fraction of the time (an average improvement of roughly 90%). We also found that the use of an automated test generation tool does not result in better fault detection in terms of mutation score compared to manual testing. Specifically, manual tests more effectively detect logical, timer and negation type of faults, compared to automatically generated tests. The results underscore the need to further study how manual testing is performed in industrial practice and the extent to which automated test generation can be used in the development of reliable systems.","PeriodicalId":112258,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125690503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 22

Towards a Testbed for Automotive Cybersecurity 迈向汽车网络安全测试平台

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST) Pub Date : 2017-03-13 DOI: 10.1109/ICST.2017.62

D. S. Fowler, Madeline Cheah, S. Shaikh, J. Bryans

引用次数: 38