E. Rodrigues, Maicon Bernardino, L. T. Costa, A. Zorzo, F. Oliveira
{"title":"PLeTsPerf - A Model-Based Performance Testing Tool","authors":"E. Rodrigues, Maicon Bernardino, L. T. Costa, A. Zorzo, F. Oliveira","doi":"10.1109/ICST.2015.7102628","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102628","url":null,"abstract":"Performance testing is a highly specialized task, since it requires that a performance engineer knows the application to be tested, its usage profile, and the infrastructure where it will execute. Moreover, it requires that testing teams expend a considerable effort and time on its automation. In this paper, we present the PLeTsPerf, a model-based performance testing tool to support the automatic generation of scenarios and scripts from application models. PLetsPerf is a mature tool, developed in collaboration with an IT company, which has been used in several works, experimental studies and pilot studies. We present an example of use to demonstrate the process of generating test scripts and scenarios from UML models to test a Web application. We also present the lessons learned and discuss our conclusions about the use of the tool.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130009639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Scalable Big Data Test Framework","authors":"Nan Li, Anthony Escalona, Yun Guo, Jeff Offutt","doi":"10.1109/ICST.2015.7102619","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102619","url":null,"abstract":"This paper identifies three problems when testing software that uses Hadoop-based big data techniques. First, processing big data takes a long time. Second, big data is transferred and transformed among many services. Do we need to validate the data at every transition point? Third, how should we validate the transferred and transformed data? We are developing a novel big data test framework to address these problems. The test framework generates a small and representative data set from an original large data set using input space partition testing. Using this data set for development and testing would not hinder the continuous integration and delivery when using agile processes. The test framework also accesses and validates data at various transition points when data is transferred and transformed.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121411745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Lightweight, Static Approach to Detecting Unbounded Thread-Instantiation Loops","authors":"Jing Xu, Yu Lei, R. Carver, D. Kung","doi":"10.1109/ICST.2015.7102579","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102579","url":null,"abstract":"In server applications, threads are created to handle incoming requests. Since threads consume significant resources including CPU cycles and memory, it is important to control the number of threads that are created. In this paper, we introduce a lightweight, static approach to detecting unbounded thread- instantiation loops that may exist in a server application. The key observation of our approach is that threads are objects of special significance and the decision logic for thread instantiation is typically not complex. Our approach checks loops against some bounded thread-instantiation patterns. A loop is considered bounded if a pattern match is found. Otherwise, it is considered unbounded. Our approach is heuristic by nature. That is, it does not guarantee to detect all the unbounded loops and may report unbounded loops that are actually bounded. To evaluate the effectiveness of our approach, we report an Eclipse plugin called ThreadBoundChecker which implements our approach and an experiment on 24 real-life Java server applications. The results of our evaluation show that our approach can effectively detect unbounded thread-instantiation loops in these applications. In particular, 12 unbounded thread-instantiation loops detected by our approach are confirmed by the original developers.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"140 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124412528","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Perspectives on White-Box Testing: Coverage, Concurrency, and Concolic Execution","authors":"Azadeh Farzan, Andreas Holzer, H. Veith","doi":"10.1109/ICST.2015.7102600","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102600","url":null,"abstract":"The last years have seen a fruitful exchange of ideas between automated software verification and white-box software testing; the industrial impact of concolic testing for sequential software is the most notable result of this interdisciplinary effort. While concolic testing is very successful at finding bugs, and even achieves verification in the limit, it is often hard to quantify the progress it achieves towards verification. In this paper, we survey two recent projects which aim to remedy this situation: In the FQL project, we devise a test specification language which facilitates precise specification of coverage criteria, and a separation of concerns between test specification and test case generation. In con2colic testing, we develop a concolic testing methodology for concurrent programs where progress is measured in terms of the data flow between program threads.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124220505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Incremental Model-Based Mutation Testing","authors":"Stefan Tiran","doi":"10.1109/ICST.2015.7102614","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102614","url":null,"abstract":"Model-based mutation testing is a promising approach to automate the generation of test cases. However, application is still limited to small models because of the computational complexity. In the proposed thesis it is investigated, how the knowledge of the internal structure of the test model can help to push the limits of the model size, which is necessary eg. for interlocking systems. A recently invented incremental language inclusion check for timed automata shall form the basis of an incremental test-case generation method based on partial test models. The approach will be applied to both synchronous and asynchronous test models. The thesis will also include hints on how to model the system-under-test in order to facilitate the proposed incremental test-case generation technique.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"276 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116292889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Integrating Concrete Symbolic Testing with an Industrial Embedded Software Development Workflow","authors":"Edgar Holleis","doi":"10.1109/ICST.2015.7102623","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102623","url":null,"abstract":"Concrete Symbolic Testing (Concolic Testing) is well established in scientific literature, but not yet widely used in industry practice. This work presents a practical approach of integrating concolic testing tools into an industrial embedded software development workflow. The practical example of a microcontroller storage subsystem serves as means to illustrate benefits and challenges.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123875611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sylvain Hallé, Nicolas Bergeron, F. Guérin, Gabriel Le Breton
{"title":"Testing Web Applications Through Layout Constraints","authors":"Sylvain Hallé, Nicolas Bergeron, F. Guérin, Gabriel Le Breton","doi":"10.1109/ICST.2015.7102635","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102635","url":null,"abstract":"The paper focuses on bugs in web applications that can be detected by analyzing the contents and layout of page elements inside a browser's window. Based on an empirical analysis of 35 real-world web sites and applications (such as Facebook, Dropbox, and Moodle), it provides a survey and classification of more than 90 instances of layout-based bugs. It then introduces Cornipickle, an automated testing tool that provides a declarative language to express desirable properties of a web application as a set of human-readable assertions on the page's HTML and CSS data. Such properties can be verified on-the-fly as a user interacts with an application.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126053714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Case Study on the Efficiency of Model-Based Testing at the European Space Agency","authors":"Stefan Mohacsi, M. Felderer, Armin Beer","doi":"10.1109/ICST.2015.7102618","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102618","url":null,"abstract":"In this paper we present the results of an empirical case study performed at the European Space Agency (ESA). In this major project, the various challenges for testing were tackled using a model-based approach for test design and the generation of executable test automation scripts. An evaluation of this approach''s efficiency identified significant cost savings and quality improvements.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"65 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114098977","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Classifying and Qualifying GUI Defects","authors":"Valéria Lelli, Arnaud Blouin, B. Baudry","doi":"10.1109/ICST.2015.7102582","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102582","url":null,"abstract":"Graphical user interfaces (GUIs) are integral parts of software systems that require interactions from their users. Software testers have paid special attention to GUI testing in the last decade, and have devised techniques that are effective in finding several kinds of GUI errors. However, the introduction of new types of interactions in GUIs (e.g., direct manipulation) presents new kinds of errors that are not targeted by current testing techniques. We believe that to advance GUI testing, the community needs a comprehensive and high level GUI fault model, which incorporates all types of interactions. The work detailed in this paper establishes 4 contributions: 1) A GUI fault model designed to identify and classify GUI faults. 2) An empirical analysis for assessing the relevance of the proposed fault model against failures found in real GUIs. 3) An empirical assessment of two GUI testing tools (i.e. GUITAR and Jubula) against those failures. 4) GUI mutants we've developed according to our fault model. These mutants are freely available and can be reused by developers for benchmarking their GUI testing tools.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"2011 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128203501","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Guided Test Generation for Finding Worst-Case Stack Usage in Embedded Systems","authors":"Tingting Yu, Myra B. Cohen","doi":"10.1109/ICST.2015.7102592","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102592","url":null,"abstract":"Embedded systems are challenging to program correctly, because they use an interrupt programming paradigm and run in resource constrained environments. This leads to a class of faults for which we need customized verification techniques. One such class of faults, stack overflows, are caused when the combination of active methods and interrupt invocations on the stack grows too large, and these can lead to data loss and other significant device failures. Developers need to estimate the worst-case stack usage (WCSU) during system design, but determining the actual maximum value is known to be a hard problem. The state of the art for calculating WCSU uses static analysis, however this has a tendency to over approximate the potential stack which can lead to wasted resources. Dynamic techniques such as random testing often under approximate the WCSU. In this paper, we present SIMSTACK, a framework that utilizes a combination of static analysis and a genetic algorithm to search for WCSUs. We perform an empirical study to evaluate the effectiveness of SIMSTACK and show that SIMSTACK is competitive with the WCSU values obtained by static analysis, and improves significantly over a random algorithm. When we use only the genetic algorithm, SIMSTACK performs almost as well as the guided technique, but takes significantly longer to converge on the maximum WCSUs.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128715020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}