{"title":"Constraints: The Future of Combinatorial Interaction Testing","authors":"J. Petke","doi":"10.1109/SBST.2015.11","DOIUrl":"https://doi.org/10.1109/SBST.2015.11","url":null,"abstract":"Combinatorial Interaction Testing (CIT) has gained a lot of attention in the area of software engineering in the last few years. CIT problems have their roots in combinatorics. Mathematicians have been concerned with the NP-complete problem of finding minimal covering arrays (in other words, minimal CIT test suites) since early nineties. With the adoption of these techniques into the area of software testing, an important gap has been identified - namely consideration of real-world constraints. We show that indeed finding an efficient way of handling constraints during search is the key factor in wider applicability of CIT techniques.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121874193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Novelty Search Approach for Automatic Test Data Generation","authors":"M. Boussaa, Olivier Barais, G. Sunyé, B. Baudry","doi":"10.1109/SBST.2015.17","DOIUrl":"https://doi.org/10.1109/SBST.2015.17","url":null,"abstract":"In search-based structural testing, metaheuristic search techniques have been frequently used to automate the test data generation. In Genetic Algorithms (GAs) for example, test data are rewarded on the basis of an objective function that represents generally the number of statements or branches covered. However, owing to the wide diversity of possible test data values, it is hard to find the set of test data that can satisfy a specific coverage criterion. In this paper, we introduce the use of Novelty Search (NS) algorithm to the test data generation problem based on statement-covered criteria. We believe that such approach to test data generation is attractive because it allows the exploration of the huge space of test data within the input domain. In this approach, we seek to explore the search space without regard to any objectives. In fact, instead of having a fitness-based selection, we select test cases based on a novelty score showing how different they are compared to all other solutions evaluated so far.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128712791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Empirical Analysis of Providing Assurance for Self-Adaptive Systems at Different Levels of Abstraction in the Face of Uncertainty","authors":"Erik M. Fredericks, B. Cheng","doi":"10.1109/SBST.2015.9","DOIUrl":"https://doi.org/10.1109/SBST.2015.9","url":null,"abstract":"Self-adaptive systems (SAS) must frequently continue to deliver acceptable behavior at run time even in the face of uncertainty. Particularly, SAS applications can self-reconfigure in response to changing or unexpected environmental conditions and must therefore ensure that the system performs as expected. Assurance can be addressed at both design time and run time, where environmental uncertainty poses research challenges for both settings. This paper presents empirical results from a case study in which search-based software engineering techniques have been systematically applied at different levels of abstraction, including requirements analysis, code implementation, and run-time validation, to a remote data mirroring application that must efficiently diffuse data while experiencing adverse operating conditions. Experimental results suggest that our techniques perform better in terms of providing assurance than alternative software engineering techniques at each level of abstraction.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"90 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131553170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Broadening the Search in Search-Based Software Testing: It Need Not Be Evolutionary","authors":"R. Feldt, Simon M. Poulding","doi":"10.1109/SBST.2015.8","DOIUrl":"https://doi.org/10.1109/SBST.2015.8","url":null,"abstract":"Search-based software testing (SBST) can potentially help software practitioners create better test suites using less time and resources by employing powerful methods for search and optimization. However, research on SBST has typically focused on only a few search approaches and basic techniques. A majority of publications in recent years use some form of evolutionary search, typically a genetic algorithm, or, alternatively, some other optimization algorithm inspired from nature. This paper argues that SBST researchers and practitioners should not restrict themselves to a limited choice of search algorithms or approaches to optimization. To support our argument we empirically investigate three alternatives and compare them to the de facto SBST standards in regards to performance, resource efficiency and robustness on different test data generation problems: classic algorithms from the optimization literature, bayesian optimization with gaussian processes from machine learning, and nested monte carlo search from game playing / reinforcement learning. In all cases we show comparable and sometimes better performance than the current state-of-the-SBST-art. We conclude that SBST researchers should consider a more general set of solution approaches, more consider combinations and hybrid solutions and look to other areas for how to develop the field.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127215625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, M. Hagiya, Yoshinori Tanabe, M. Yamamoto
{"title":"GRT at the SBST 2015 Tool Competition","authors":"Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, M. Hagiya, Yoshinori Tanabe, M. Yamamoto","doi":"10.1109/SBST.2015.19","DOIUrl":"https://doi.org/10.1109/SBST.2015.19","url":null,"abstract":"GRT (Guided Random Testing) is an automatic test generation tool for Java code, which leverages static and dynamic program analysis to guide run-time test generation. In this paper, we summarize competition results and experiences of GRT in participating in SBST 2015, where GRT ranked first with a score of 203.73 points over 63 Java classes from 10 packages of 9 open-source software projects.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116359854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Annibale Panichella, Fitsum Meshesha Kifetew, P. Tonella
{"title":"Results for EvoSuite -- MOSA at the Third Unit Testing Tool Competition","authors":"Annibale Panichella, Fitsum Meshesha Kifetew, P. Tonella","doi":"10.1109/SBST.2015.14","DOIUrl":"https://doi.org/10.1109/SBST.2015.14","url":null,"abstract":"Evo Suite-MOSA is a unit test data generation tool that employs a novel many-objective optimization algorithm suitably developed for branch coverage. It was implemented by extending the Evo Suite test data generation tool. In this paper we present the results achieved by Evo Suite-MOSA in the third Unit Testing Tool Competition at SBST'15. Among six participants, Evo Suite-MOSA stood third with an overall score of 189.22.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122041561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maurizio Leotta, Andrea Stocco, F. Ricca, P. Tonella
{"title":"Meta-heuristic Generation of Robust XPath Locators for Web Testing","authors":"Maurizio Leotta, Andrea Stocco, F. Ricca, P. Tonella","doi":"10.1109/SBST.2015.16","DOIUrl":"https://doi.org/10.1109/SBST.2015.16","url":null,"abstract":"Test scripts used for web testing rely on DOM locators, often expressed as XPaths, to identify the active web page elements and the web page data to be used in assertions. When the web application evolves, the major cost incurred for the evolution of the test scripts is due to broken locators, which fail to locate the target element in the new version of the software. We formulate the problem of automatically generating robust XPath locators as a graph exploration problem, for which we provide an optimal, greedy algorithm. Since such an algorithm has exponential time and space complexity, we present also a genetic algorithm.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133435751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kevin Salvesen, Juan P. Galeotti, F. Gross, G. Fraser, A. Zeller
{"title":"Using Dynamic Symbolic Execution to Generate Inputs in Search-Based GUI Testing","authors":"Kevin Salvesen, Juan P. Galeotti, F. Gross, G. Fraser, A. Zeller","doi":"10.1109/SBST.2015.15","DOIUrl":"https://doi.org/10.1109/SBST.2015.15","url":null,"abstract":"Search-based testing has been successfully applied to generate complex sequences of events for graphical user interfaces (GUIs), but typically relies on simple heuristics or random values for data widgets like text boxes. This may greatly reduce the effectiveness of test generation for applications which expect specific input values to be entered in their GUI by users. Generating such specific input values is one of the virtues of dynamic symbolic execution (DSE), but DSE is less suitable to generate sequences of events. Therefore, this paper describes a hybrid approach that uses search-based testing to generate sequences of events, and DSE to build input data for text boxes. This is achieved by replacing standard widgets in a system under test with symbolic ones, allowing us to execute GUIs symbolically. In this paper, we demonstrate an extension of the search-based GUI testing tool EXSYST, which uses DSE to successfully increase the obtained code coverage on two case study applications.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122583371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"EvoSuite at the SBST 2015 Tool Competition","authors":"G. Fraser, Andrea Arcuri","doi":"10.1109/SBST.2015.13","DOIUrl":"https://doi.org/10.1109/SBST.2015.13","url":null,"abstract":"EvoSuite is a mature research prototype that automatically generates unit tests for Java code. This paper summarizes the results and experiences of Evo Suite's participation at the third unit testing competition at SBST 2015. An unfortunate issue of conflicting dependency versions in two out of the nine benchmark projects reduced Evo Suite's overall score to 190.6, leading to the overall second rank.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121735911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Unit Testing Tool Competition -- Round Three","authors":"U. Rueda, T. Vos, I.S.W.B. Prasetya","doi":"10.1109/SBST.2015.12","DOIUrl":"https://doi.org/10.1109/SBST.2015.12","url":null,"abstract":"This paper describes the third round of the Java Unit Testing Tool Competition. This edition of the contest evaluates no less than seven automated testing tools! And, like during the second round, test suites written by human testers are also used for comparison. This paper contains the full results of the evaluation.","PeriodicalId":219232,"journal":{"name":"2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131703916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}