M. Vieira, Johanne Leduc, W. Hasling, R. Subramanyan, Jürgen Kazmeier
{"title":"Automation of GUI testing using a model-driven approach","authors":"M. Vieira, Johanne Leduc, W. Hasling, R. Subramanyan, Jürgen Kazmeier","doi":"10.1145/1138929.1138932","DOIUrl":"https://doi.org/10.1145/1138929.1138932","url":null,"abstract":"This paper describes an ongoing research on test case generation based on Unified Modeling Language (UML). The described approach builds on and combines existing techniques for data and graph coverage. It first uses the Category-Partition method to introduce data into the UML model. UML Use Cases and Activity diagrams are used to respectively describe which functionalities should be tested and how to test them. This combination has the potential to create a very large number of test cases. This approach offers two ways to manage the number of tests. First, custom annotations and guards use the Category-Partition data which allows the designer tight control over possible, or impossible, paths. Second, automation allows different configurations for both the data and the graph coverage. The process of modeling UML activity diagrams, annotating them with test data requirements, and generating test scripts from the models is described. The goal of this paper is to illustrate the benefits of our model-based approach for improving automation on software testing. The approach is demonstrated and evaluated based on use cases developed for testing a graphical user interface (GUI).","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126318685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A survey of coverage based testing tools","authors":"Qian Yang, Juan Li, D. Weiss","doi":"10.1145/1138929.1138949","DOIUrl":"https://doi.org/10.1145/1138929.1138949","url":null,"abstract":"Test coverage is sometimes used as a way to measure how thoroughly software is tested. Coverage is used by software developers and sometimes by vendors to indicate their confidence in the readiness of their software. This survey studies and compares 17 coverage-based testing tools focusing on, but not restricted to coverage measurement. We also survey additional features, including program prioritization for testing, assistance in debugging, automatic generation of test cases, and customization of test reports. Such features make tools more useful and practical, especially for large-scale, real-life commercial software applications. Our initial motivations were both to understand the available test coverage tools and to compare them to a tool that we have developed, called eXVantage1 (a tool suite that includes code coverage testing, debugging, performance profiling, and reporting). Our study shows that each tool has its unique features tailored to its application domains. Therefore this study can be used to pick the right coverage testing tools depending on various requirements.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"257 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121310857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Joseph C. Okika, A. Ravn, Zhiming Liu, Lokesh Siddalingaiah
{"title":"Developing a TTCN-3 test harness for legacy software","authors":"Joseph C. Okika, A. Ravn, Zhiming Liu, Lokesh Siddalingaiah","doi":"10.1145/1138929.1138950","DOIUrl":"https://doi.org/10.1145/1138929.1138950","url":null,"abstract":"We describe a prototype test harness for an embedded system which is the control software for a modern marine diesel engine. The operations of such control software requires complete certification. We adopt Testing and Test Control Notation (TTCN-3) to define test cases for this purpose. The main challenge in developing the test harness is to interface a generic test driver to the legacy software and provide a suitable interface for test engineers. The main contribution of this paper is a demonstration of a suitable design for such a test harness. It includes: a TTCN-3 test driver in C++, the legacy control software in C, a Graphical User Interface (GUI)and the connectors in Java. Our experience shows that it is feasible to use TTCN-3 in developing a test harness for a legacy software for an embedded system, even when it involves different heterogeneous components.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128509789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Economic perspectives in test automation: balancing automated and manual testing with opportunity cost","authors":"R. Ramler, Klaus Wolfmaier","doi":"10.1145/1138929.1138946","DOIUrl":"https://doi.org/10.1145/1138929.1138946","url":null,"abstract":"Testing is a major cost factor in software development. Test automation has been proposed as one solution to reduce these costs. Test automation tools promise to increase the number of tests they run and the frequency at which they run them. So why not automate every test? In this paper we discuss the question \"When should a test be automated?\" and the trade-off between automated and manual testing. We reveal problems in the overly simplistic cost models commonly used to make decisions about automating testing. We introduce an alternative model based on opportunity cost and present influencing factors on the decision of whether or not to invest in test automation. Our aim is to stimulate discussion about these factors as well as their influence on the benefits and costs of automated testing in order to support researchers and practitioners reflecting on proposed automation approaches.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133431284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A strategy to perform coverage testing of mobile applications","authors":"M. Delamaro, A. Vincenzi, J. Maldonado","doi":"10.1145/1138929.1138952","DOIUrl":"https://doi.org/10.1145/1138929.1138952","url":null,"abstract":"The development of wireless application has recently received more attention due to the increment in the number and in the power of mobile devices such as PDA's and cellular phones. Different methods and techniques have been developed to ease the design and development of applications for these kind of devices. Also, different languages have been proposed to provide support for such platform, such as J2ME and Brew. On the other hand, few attention has been given to testing activity in this scenario. Some works try to test the functional aspects of a given application, others try to perform load, usability and stress testing. In this article we present a strategy to support coverage testing for mobile device software in such a way that the applications can be tested not only on emulators, but also on their real target mobile devices with the aid of structural coverage assessment. We also present an environment which supports the proposed strategy. Such environment is implemented in a tool, named JaBUTi/ME. A simple case illustrating how JaBUTi/ME can be used is also presented.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125475558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Coupling-based class integration and test order","authors":"Aynur Abdurazik, A. Offutt","doi":"10.1145/1138929.1138940","DOIUrl":"https://doi.org/10.1145/1138929.1138940","url":null,"abstract":"During component-based and object-oriented software development, software classes exhibit relationships that complicate integration, including method calls, inheritance, and aggregation. When classes are integrated and tested, an order of integration must be established. The difficulty arises when cyclic dependencies exist - the functionality that is used by the first class to be tested must be mimicked by creating \"stubs\" (sometimes called \"mocks\"), an expensive and error-prone operation. This problem is generally called the class integration and test order (CITO) problem, and solutions must be fully automated for integration and testing to proceed smoothly and efficiently. This paper describes new techniques and algorithms to solve the CITO problem. New results include improved edge weights that are derived from quantitative coupling measures to more precisely model the cost of stubbing, and the use of weights on nodes, allowing more information to be used. Also, a new algorithm for computing the integration and test orders is presented. The technique is compared with an existing approach with positive results.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"229 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122116992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Advanced unit testing: how to scale up a unit test framework","authors":"Cyrille Artho, Armin Biere","doi":"10.1145/1138929.1138947","DOIUrl":"https://doi.org/10.1145/1138929.1138947","url":null,"abstract":"Unit testing is a scalable and effective way to uncover software faults. In the JNuke project, automated regression tests combined with coverage measurement ensured high code quality throughout the project. By using a custom testing environment, functionality was extended beyond what is commonly available by unit test frameworks. Low-overhead memory leak detection was implemented through wrapping. Automated support for log files made it possible to track the internal state of objects, which is often much more expedient than writing test code. These extensions allowed the easy-to-use unit test framework to scale up to large-scale tests. The techniques can be ported to existing test frameworks.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125662047","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Substra: a framework for automatic generation of integration tests","authors":"Hai Yuan, Tao Xie","doi":"10.1145/1138929.1138942","DOIUrl":"https://doi.org/10.1145/1138929.1138942","url":null,"abstract":"A component-based software system consists of well-encapsulated components that interact with each other via their interfaces. Software integration tests are generated to test the interactions among different components. These tests are usually in the form of sequences of interface method calls. Although many components are equipped with documents that provide informal specifications of individual interface methods, few documents specify component interaction constraints on the usage of these interface methods, including the order in which these methods should be called and the constraints on the method arguments and returns across multiple methods. In this paper, we propose Substra, a framework for automatic generation of software integration tests based on call-sequence constraints inferred from initial-test executions or normal runs of the subsystem under test. Two types of sequencing constraints are inferred: shared subsystem states and object define-use relationships. The inferred constraints are used to guide automatic generation of integration tests. We have implemented Substra with a tool and applied the tool on an ATM example. The preliminary results show that the tool can effectively generate integration tests that exercise new program behaviors.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123969759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Cai, Yong-Chao Li, Wei-Yi Ning, W. E. Wong, Hai Hu
{"title":"Optimal and adaptive testing with cost constraints","authors":"K. Cai, Yong-Chao Li, Wei-Yi Ning, W. E. Wong, Hai Hu","doi":"10.1145/1138929.1138944","DOIUrl":"https://doi.org/10.1145/1138929.1138944","url":null,"abstract":"This paper generalizes our previous work on optimal and adaptive testing to consider a more general scenario of software testing resource constraints. The assumption is that software testing must be stopped once the allowed testing resources are used up. The contributions of this paper are as follows. First, we show that software testing with fixed resource constraints can be handled in the framework of the controlled Markov chains (CMC) approach to software testing. Second, an algorithm is adopted to reduce the computational complexity of on-line decision making in optimal and adaptive testing. Finally, the simulation results presented in this paper further confirm the effectiveness of the idea of adaptive testing in particular, and that of software cybernetics (which explores the interplay between software and control) in general.","PeriodicalId":443108,"journal":{"name":"International Conference/Workshop on Automation of Software Test","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125864674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}