{"title":"Optimizing IPOG's Vertical Growth with Constraints Based on Hypergraph Coloring","authors":"Feng Duan, Yu Lei, L. Yu, R. Kacker, D. R. Kuhn","doi":"10.1109/ICSTW.2017.37","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.37","url":null,"abstract":"In this paper, we present an optimization of IPOG's vertical growth phase in the presence of constraints. The vertical growth problem is modeled as a classical NP-hard graph problem called \"Minimum Vertex Coloring\". In the graph model, vertices are either missing tuples that are waiting to be colored or existing tests that are already colored in different colors at the initial state, edges/hyperedges are conflicts among vertices that cannot be put in a same test. After coloring, a group of vertices in same color can be transformed to exactly a valid test. Since the original IPOG algorithm uses an arbitrary order to cover missing tuples during vertical growth, in order to reduce the number of tests, we compute the Degree of Conflicts (DOC) for each tuple, and cover the tuples in the non-increasing order of DOC. We implement a new IPOG algorithm incorporating this optimization. The experimental results show that the new IPOG algorithm reduces the number of tests for many real-life systems with constraints.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130799863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the ITEQS 2017 Chairs","authors":"Mehrdad Saadatmand, B. Lindström, M. Bohlin","doi":"10.1109/ICSTW.2017.14","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.14","url":null,"abstract":"Welcome to the 1st International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems (ITEQS 2017), which is co-located with the 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017) in Tokyo, Japan. Testing extra-functional properties is often a challenge since such properties are emergent and traditional testing techniques therefore may not be applicable. ITEQS aims to gather researchers and practitioners to discuss these fundamental challenges and to exchange ideas on how to address them.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"41 21","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113974438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Software Testing Design Techniques Used in Automated Vehicle Simulations","authors":"Satoshi Masuda","doi":"10.1109/ICSTW.2017.55","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.55","url":null,"abstract":"Research and development in the field of automated vehicles has increased along with related works about its software. Software testing in automated vehicles is key to launching safe and reliable vehicles. Several issues in the software testing of automated vehicles have been raised including extremely large space of test input, the high cost of test executions in a physical environment, test oracles not being simple Boolean properties, and so on. Automated vehicle simulations are a solution of cost reduction in test execution. However, space of test input is extremely large. Extremely large space of test input comes from sensing data. In this paper, we discuss the software architecture of automated vehicle simulation as the target software. We raise issues on software testing in these simulations on the basis of the related works. We then discuss test design techniques used in automated vehicle simulations.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"204 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131803514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vinicius H. S. Durelli, Nilton M. De Souza, M. Delamaro
{"title":"Are Deletion Mutants Easier to Identify Manually?","authors":"Vinicius H. S. Durelli, Nilton M. De Souza, M. Delamaro","doi":"10.1109/ICSTW.2017.32","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.32","url":null,"abstract":"Mutation testing changes a program by applying mutation operators, which generate many slightly different versions of the original program (i.e., mutants). Identifying which mutants are equivalent to the original program is one of the major barriers to a wider adoption of mutation testing. Recent studies suggest that an effective approach to mitigating this problem is to use only a subset of all mutation operators: some operators result in few mutants, but these mutants yield tests that are practically as effective as using all mutants. One research direction has been to investigate the use of only one type of operator: deletion mutation operators. Deletion operators are deemed to be effective and, based on an intuitive notion, we conjectured that these operators lead to mutants that are easier to manually assess for equivalence. Thus, we set out to compare the (human) cost of detecting the equivalence of mutants generated by deletion operators with the cost of analyzing mutants created by traditional operators. To this end, we designed an experiment and conducted it with 12 graduate students. This study investigated two proxies for the cost of identifying equivalent mutants by hand: (i) number of mistakes made by the subjects and (ii) the length of time spent analyzing mutants. According to the results of our experiment, there is no significant difference between the costs of manually assessing traditional and deletion operators for equivalence. We also probed into the cost of each operator and found that CCDL is the most costly deletion operator. Another core finding is that ORRN, ORLN, and ORSN account for the lion's share of the cost involved in manually assessing traditional mutants for equivalence.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134644373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Johannes Bräuer, Reinhold Plösch, Manuel Windhager
{"title":"Are CISQ Reliability Measures Practical? A Research Perspective","authors":"Johannes Bräuer, Reinhold Plösch, Manuel Windhager","doi":"10.1109/ICSTW.2017.7","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.7","url":null,"abstract":"The Object Management Group (OMG), which is driven by industry, proposes an operational standard for measuring reliability by providing specifications for 29 reliability measures. The goal of this article is to systematically assess whether (1) the provided measurement specifications are suitable to be implemented in a static testing tool used in practice and (2) the measures are suitable for capturing reliability issues in software. Therefore, we implemented the CISQ measures (CISQ is the consortium responsible for driving the quality related topics of the OMG) for Java in our quality measurement tool after defining assumptions resulting from the language independent and at some point imprecise specifications. In format of a case study, the CISQ-based measurement tool has then been applied on several versions of the open source project HSQLDB. The results show that CISQ measures properties that are vital for fulfilling reliability requirements. In the course of the case study, the engineers of HSQLDB fixed a number of issues identified by our tooling as they were considered to be critical. While a number of rule violations are considered to be still problematic, they could not be fixed since the engineers did not have the code ownership. In these cases, they proposed improvement suggestions to the responsible teams.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129011173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Test Optimization Using Combinatorial Test Design: Real-World Experience in Deployment of Combinatorial Testing at Scale","authors":"Saritha Route","doi":"10.1109/ICSTW.2017.50","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.50","url":null,"abstract":"Clients today want more for less and the IBM test mantra of Test Less Test Right helps address this ask by placing Combinatorial Test Design (CTD) at the heart of the solution. This document presents two case studies of CTD implementation in client engagements and focuses on the approach, process and challenges addressed to scale up the implementation and make CTD a mainstream activity. The IBM Focus tool was used in both cases to implement Combinatorial Test Design for optimization of tests and for reducing test effort while increasing test coverage.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126616634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Framework for Model-Based Design and Verification of Human-in-the-Loop Cyber-Physical Systems","authors":"Filip Cuckov, Grant Rudd, L. Daly","doi":"10.1109/ICSTW.2017.77","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.77","url":null,"abstract":"We present a framework for comprehensive model-based systematic design and staged verification of Human-in-the-Loop Cyber-Physical Systems (HiLCPS) to handle their inherent complexity. HiLCPS are systems where humans are in the middle of the feedback loop between their cyber and the physical components. HiLCPS designers require modern tools and simpler model-based approaches for design and verification which encapsulate human behavior and its effect on the system.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"71 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115717381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using Model-Checking for Timing Verification in Industrial System Design","authors":"L. Rioux, R. Henia, Nicolas Sordon","doi":"10.1109/ICSTW.2017.68","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.68","url":null,"abstract":"To reduce the risk of costly timing errors detected after the integration phase, there is a need for efficient, reliable and automated timing prediction methods applicable already at the early design stages, even if based on incomplete or partial timing information. The obtained timing estimations can be consolidated along the different design phases, until the final integration. Model-Checking is a good candidate for coarse grained timing verification, since, for such use, it does scale and provide exact estimates. However, its use in the industry is conditioned by a seamless integration in the development process. In this paper, we present our solution to connect the model-checking tool ROMEO to the design process, thus allowing the verification of the system timing behavior at early stages.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115765071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Iftekhar Ahmed, Carlos Jensen, Alex Groce, P. McKenney
{"title":"Applying Mutation Analysis on Kernel Test Suites: An Experience Report","authors":"Iftekhar Ahmed, Carlos Jensen, Alex Groce, P. McKenney","doi":"10.1109/ICSTW.2017.26","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.26","url":null,"abstract":"Mutation analysis is an established technique for measuring the completeness and quality of a test suite. Despite four decades of research on this technique, its use in large systems is still rare, in part due to computational requirements and high numbers of false positives. We present our experiences using mutation analysis on the Linux kernel's RCU (Read Copy Update) module, where we adapt existing techniques to constrain the complexity and computation requirements. We show that mutation analysis can be a useful tool, uncovering gaps in even well-tested modules like RCU. This experiment has so far led to the identification of 3 gaps in the RCU test harness, and 2 bugs in the RCU module masked by those gaps. We argue that mutation testing can and should be more extensively used in practice.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128445278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Messages from the InSTA 2017 Chairs","authors":"S. Masuda","doi":"10.1109/ICSTW.2017.51","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.51","url":null,"abstract":"Messages from the Chairs - InSTA 2017.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133442419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}