K. Larsen, Florian Lorber, B. Nielsen, Ulrik Nyman
{"title":"Mutation-Based Test-Case Generation with Ecdar","authors":"K. Larsen, Florian Lorber, B. Nielsen, Ulrik Nyman","doi":"10.1109/ICSTW.2017.60","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.60","url":null,"abstract":"Model-based testing is a well-known technique for automating the otherwise tedious process of testing. Test cases are automatically created from a formal model, according to some test criterion which determines when the test suite is complete. In model-based mutation testing, the test criterion is defined via faulty models, called mutants, which are used to create test cases that specifically target the modeled faults. To be able to reveal timing related faults, timed automata can be used as the test model. While model-based mutation testing has already been applied to timed automata, we show how to implement the technique more efficiently with the tool Ecdar, which belongs to the well-known UPPAAL tool family. The tool is used to perform an unbounded conformance check between the correct specification and the mutants, based on a notion of timed refinement. If a mutant does not refine the specification, Ecdar creates a strategy for reaching the non-conformance, which can be used as an adaptive test case. We applied the procedure to the timed automata model of a car alarm system, which was used in the previous approach of model-based mutation testing for timed automata, and compare the two approaches based on the results.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115957132","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reducing Mutants with Mutant Killable Precondition","authors":"Chihiro Iida, Shingo Takada","doi":"10.1109/ICSTW.2017.29","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.29","url":null,"abstract":"Mutation analysis is a method for predicting the quality of test suite accurately. However, it has high computational cost due to the number of mutants that are generated. For example, the ROR (Relational Operator Replacement) mutation operator will generate seven mutants for just one relational operator. Naively applying multiple operators over the entire program can result in a high number of generated mutants. One way to reduce the number of mutants is to omit redundant mutants. In this paper, we propose an approach to reducing mutants by using mutant killable precondition to identify redundant mutants. A mutant killable precondition is a logical expression for killing a mutant. We focus on the conditional expression for control flow statements, such as if and while statements. We describe the mutant killable precondition for conditional expressions that compare numbers, e.g., x > 0. We then discuss mutants that are generated for such conditional expressions, and find the minimal set of mutants. Finally, we show the theoretical and empirical reduction rate of our approach.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124745409","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reducing the Concretization Effort in FSM-Based Testing of Software Product Lines","authors":"V. H. Fragal, A. Simão, A. T. Endo, M. Mousavi","doi":"10.1109/ICSTW.2017.61","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.61","url":null,"abstract":"To test a Software Product Line (SPL), the test artifacts and the techniques must be extended to support variability. In general, when new SPL products are developed, more tests are generated to cover new or modified features. A dominant source of extra effort for such tests is the concretization of newly generated tests. Thus, minimizing the amount of new nonconcretized tests required to perform conformance testing on new products reduces the overall test effort. In this paper, we propose a test reuse strategy for conformance testing of SPL products that aims at reducing test effort. We use incremental test generation methods based on finite state machines (FSMs) to maximize test reuse. We combine these methods with a selection algorithm used to identify non-redundant concretized tests. We illustrate our strategy using examples and a case study with an embedded mobile SPL. The results indicate that our strategy can save up to 36% of test effort in comparison to current test reuse strategies for the same fault detection capability.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125192383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Closing the Gap between Unit Test Code and Documentation","authors":"Karsten Stocker, H. Washizaki, Y. Fukazawa","doi":"10.1109/ICSTW.2017.56","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.56","url":null,"abstract":"Test Driven Development as well as the documentation of tests and their architecture are today an important pillar of software quality assurance. The change of requirements during the implementation phase entails a need to change tests as well as the test documentation of the software. Since unit tests are specified in the implementation language, an interdisciplinary readable documentation must be maintained, which is structurally easier to comprehend and also make the test transparent for persons who are not involved into code writing. This leads to additional effort, costs and possibly inconsistencies between the test and its documentation. This gap in the workflow could be closed by Tanni – a domain specific language, which allows the specification of test cases in the form of interdisciplinary readable tables without requiring programming skills. Based on them executable test code for the respective unit test framework is generated. This merges specification and documentation of unit test cases to one step of work. By this the mentioned additional effort, costs and imminent inconsistencies can be reduced. The Language Workbench Meta Programming System from JetBrains serves as a technological base and is enabler for further positive effects which possibly could be gained by using the described language.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129603714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Keisuke Watanabe, T. Fukamachi, Naoyasu Ubayashi, Yasutaka Kamei
{"title":"Automated A/B Testing with Declarative Variability Expressions","authors":"Keisuke Watanabe, T. Fukamachi, Naoyasu Ubayashi, Yasutaka Kamei","doi":"10.1109/ICSTW.2017.72","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.72","url":null,"abstract":"A/B testing is the experiment strategy, which is often used on web or mobile application development. In A/B testing, a developer has to implement multiple variations of application, assign each variation to a subset of the entire user population randomly, and analyze log data to decide which variation should be used as a final product. Therefore, it is challenging to keep the application code clean in A/B testing, because defining variations of software or assigning user to each variation needs the modification of code. In fact there are some existing tools to approach this problem. Considering such a context of A/B testing research, we propose the solution based on the interface Archface-U and AOP (Aspect Oriented Programming) which aims to minimize the complication of code in A/B testing.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133305534","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alexandre Langeois, E. Almeida, Anthony Ventresque
{"title":"BDTest, a System to Test Big Data Frameworks","authors":"Alexandre Langeois, E. Almeida, Anthony Ventresque","doi":"10.1109/ICSTW.2017.75","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.75","url":null,"abstract":"Testing Big Data Processing systems is a challenging task as these systems are usually distributed on various virtual machines (potentially hosted by remote servers). In this poster we present a platform for testing non-functional properties of Big Data framework and a first implementation with Hadoop, a well known big data management and processing platform.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121055659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas Laurent, L. Guillot, Motomichi Toyama, Ross Smith, Dan Bean, Anthony Ventresque
{"title":"Towards a Gamified Equivalent Mutants Detection Platform","authors":"Thomas Laurent, L. Guillot, Motomichi Toyama, Ross Smith, Dan Bean, Anthony Ventresque","doi":"10.1109/ICSTW.2017.70","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.70","url":null,"abstract":"This poster presents a gamified system for equivalent mutants detection. This system can be used as a standalone tool for developers and testing teams alike - but we plan to use this system on a crowdsourcing platform to evaluate the various parameters involved in the detection of equivalent mutants, such as, expertise (coding and testing), familiarity with the code base, complexity of the code and tests, measured likelihood of equivalent mutants.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132589231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"MutRex: A Mutation-Based Generator of Fault Detecting Strings for Regular Expressions","authors":"Paolo Arcaini, A. Gargantini, E. Riccobene","doi":"10.1109/ICSTW.2017.23","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.23","url":null,"abstract":"Regular expressions (regexes) permit to describe set of strings using a pattern-based syntax. Writing a correct regex that exactly captures the desired set of strings is difficult, also because a regex is seldom syntactically incorrect, and so it is rare to detect faults at parse time. We propose a fault-based approach for generating tests for regexes. We identify fault classes representing possible mistakes a user can make when writing a regex, and we introduce the notion of distinguishing string, i.e., a string that is able to witness a fault. Then, we provide a tool, based on the automata representation of regexes, for generating distinguishing strings exposing the faults introduced in mutated versions of a regex under test. The basic generation process is improved by two techniques, namely monitoring and collecting. Experiments show that the approach produces compact test suites having a guaranteed fault detection capability, differently from other test generation approaches.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"513 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122884629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Generating Controllably Invalid and Atypical Inputs for Robustness Testing","authors":"Simon M. Poulding, R. Feldt","doi":"10.1109/ICSTW.2017.21","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.21","url":null,"abstract":"One form of robustness in a software system is its ability to handle, in an appropriate manner, inputs that are unexpected compared to those it would experience in normal operation. In this paper we investigate a generic approach to generating such unexpected test inputs by extending a framework that we have previously developed for the automated creation of complex and high-structured test data. The approach is applied to the generation of valid inputs that are atypical as well as inputs that are invalid. We demonstrate that our approach enables control of the 'degree' to which the test data is invalid or atypical, and show empirically that this can alter the extent to which the robustness of a software system is exercised during testing.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129091335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Composition-Based Method for Combinatorial Test Design","authors":"A. Zamansky, Amir Shwartz, Seri Khoury, E. Farchi","doi":"10.1109/ICSTW.2017.45","DOIUrl":"https://doi.org/10.1109/ICSTW.2017.45","url":null,"abstract":"The notion of composition in the context of combinatorial test design naturally arises in situations where the output of one module is used as an input to another module. Composability of tests plans has several important benefits as it facilitates both test and model reuse in such sequential scenarios. In this paper we describe a method for an automatic construction of composable tests plans, one by-product of which is a test plan for the SUT, obtained by their composition. The proposed method is implemented using binary decision diagrams (BDDs), which allows for an efficient and scalable composition-oriented CTD for real-life testing problems. We evaluate the method on a case study of a composition of two Unix Shell operations.","PeriodicalId":312017,"journal":{"name":"2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116475850","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}