2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)最新文献

{"title":"Conceptualization and Evaluation of Component-Based Testing Unified with Visual GUI Testing: An Empirical Study","authors":"Emil Alégroth, Zebao Gao, R. Oliveira, A. Memon","doi":"10.1109/ICST.2015.7102584","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102584","url":null,"abstract":"In this paper we present the results of a two-phase empirical study where we evaluate and compare the applicability of automated component-based Graphical User Interface (GUI) testing and Visual GUI Testing (VGT) in the tools GUITAR and a prototype tool we refer to as VGT GUITAR. First, GUI mutation operators are defined to create 18 faulty versions of an application on which both tools are then applied in an experiment. Results from 456 test case executions in each tool show, with statistical significance, that the component-based approach reports more false negatives than VGT for acceptance tests but that the VGT approach reports more false positives for system tests. Second, a case study is performed with larger open source applications, ranging from 8,803-55,006 lines of code. Results show that GUITAR is applicable in practice but has some challenges related to GUI component states. The results also show that VGT GUITAR is currently not applicable in practice and therefore requires further research and development. Based on the study's results we present areas of future work for both test approaches and conclude that the approaches have different benefits and drawbacks. The component-based approach is robust and executes tests faster than the VGT approach, with a factor of 3. However, the VGT approach can perform visual assertions and is perceived more flexible than the component- based approach. These conclusions let us hypothesize that a combination of the two approaches is the most suitable in practice and therefore warrants future research.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"88 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116781870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"U-Test: Evolving, Modelling and Testing Realistic Uncertain Behaviours of Cyber-Physical Systems","authors":"Shaukat Ali, T. Yue","doi":"10.1109/ICST.2015.7102637","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102637","url":null,"abstract":"Uncertainty is intrinsic in Cyber-Physical Systems (CPSs) due to novel interactions of embedded systems, networking equipment, cloud infrastructures and humans. Our daily life has been increasing dependent on CPS applications in safety/mission critical domains such as healthcare, aerospace, oil/gas and maritime. For example, the National Institute of Standards and Technology (NIST) reported that direct CPS applications account for more than $32.3 trillions and expect to grow $82 trillions by 2025 (about half of the world economy). Expecting enormous dependence of our lives on CPSs in the future, dealing with uncertainty at an acceptable cost is vital to avoid posing undue threats to its users and environment. To ensure correct delivery of their functions at an acceptable cost even in the presence of uncertainty, CPSs must be reliable, robust, efficient, safe, and secure. All these properties are facets of a more general property often known as dependability. Improving system dependability first and foremost relies on the ability to verify and validate CPSs in a cost-effective manner and one way of achieving this is via systematic and automated Model-Based Testing (MBT): automated derivation of test cases from a behavioral model of a system. MBT supports rigorous, systematic, and automated testing, which eventually reduces the number of faults in the delivered systems and thus improves their quality. The goal of the U-Test project (a recently funded project under the EU Horizon2020 program (http://ec.europa.eu/programmes/horizon2020/) is to improve the dependability of CPSs, via cost-effective, model-based and search-based testing of CPSs under unknown risky uncertainty. Unknown uncertainty is the state of a CPS that can only be determined at the runtime as opposed to known uncertainty that is known at the design time and outcome from risky uncertainty is undesirable. To achieve our goal, we will advance the current state-of-art of testing CPSs by developing a novel solution based on sound theoretical foundation for uncertainty testing in the following steps: 1) Developing a light-weight modelling solution with rich formalism to support minimal modelling of known uncertainty with risk information; 2) Intelligently evolving known uncertainty models towards realistic and risky unknown uncertainty models (evolved models) using search algorithms (e.g., genetic algorithms mimicking natural selection); and 3) Automatically generating test cases from the evolved models to test a CPS under unknown uncertainty to ensure that the CPS continues to operate properly and possibly at a reduced quality of operation, rather than failing completely.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134008035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Virtual Bank for Development and Testing","authors":"Daniel Kunzle, Carl F. Worms","doi":"10.1109/ICST.2015.7102620","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102620","url":null,"abstract":"This paper describes how a Virtual Bank - a development and testing environment based on a purely synthetic test data base - has been established to address the challenge of protecting sensitive business and client data, while at the same time collaborating in application development and testing across country and jurisdiction borders.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128944738","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Mobile Testing in Software Industry Using Agile: Challenges and Opportunities","authors":"Andreia Santos, Igor Correia","doi":"10.1109/ICST.2015.7102625","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102625","url":null,"abstract":"The use of mobile devices grows significantly in all situations of day-to-day. The intense rhythm of the mobile device market has been pushing the use of practical software development and accelerated implementation. Both development activities hand the testing for these applications has proven challenging. A number of reasons, indicate that conventional software testing efforts are not successful in mobile applications. The testing automation approach for mobile applications, in most of the cases, does not cover all of the application features, which makes the manual test using devices necessary. The challenge is even greater in projects with agile methodologies, where test automation is one of core parts of the development process. In this work, the main objective is the sharing of the biggest challenges that are faced, especially during testing of mobile applications from the point of view of product quality; as well as the same may be supplied with the collaboration of the entire team in an agile development environment. For this purpose we will use a real project with NFC technology as example.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116341003","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Yes! You Can Use Your Model Checker to Verify OSEK/VDX Applications","authors":"Haitao Zhang, Toshiaki Aoki, Yuki Chiba","doi":"10.1109/ICST.2015.7102612","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102612","url":null,"abstract":"OSEK/VDX, a standard of automobile OS, has been widely adopted by many manufacturers to design and develop a vehicle-mounted OS. With the increasing functionalities in vehicles, more and more complex applications are developed based on the OSEK/VDX OS. However, how to ensure the reliability of developed applications is becoming a challenge for developers. As to ensure the reliability of developed applications, model checking as an exhaustive technique can be applied to verify the OSEK/VDX applications. There exist many model checkers that have been successfully applied to verify sequential software and general multi-threaded software. However, it is hard to directly use existing model checkers to precisely verify OSEK/VDX applications, since the execution characteristics of OSEK/VDX applications are different from the sequential software and general multi-threaded software. In this paper, we describe and develop an approach to translate OSEK/VDX applications into sequential programs in order to employ existing model checkers to precisely verify OSEK/VDX applications. The value of our approach is that it can be considered as a front-end translator for enabling existing model checkers to verify OSEK/VDX applications.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"107 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121943891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MetTest: A Test Automation Framework for Development of a Point-To-Multipoint Radio","authors":"M. Tokcan, Onur Ozturk, Hakan Tuna","doi":"10.1109/ICST.2015.7102624","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102624","url":null,"abstract":"Continuous and timely testing is an indispensable activity for agile software development, which requires assignment of persistent and dedicated resources throughout the whole life cycle of a project. Testing becomes even more complicated when the system under test is a distributed and embedded real-time system. In this study, we present MetTest, an industrial test automation framework designed and deployed for development of a point-to-multipoint Ethernet based radio having various modes of operation, each with an associated performance goal. MetTest pursues an automated and a scheduled sequence of actions consisting of building executable from a code repository, running unit and full system performance tests on hardware, measuring and verifying necessary performance metrics and e- mailing a summary to the attention of software developers. MetTest also stores test results for long term analysis and tracking of software performance through a web application. Some long- term statistics and test results acquired from the framework are also presented to demonstrate the usefulness of the framework.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130136950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring Test Suite Diversification and Code Coverage in Multi-Objective Test Case Selection","authors":"Debajyoti Mondal, H. Hemmati, Stephane Durocher","doi":"10.1109/ICST.2015.7102588","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102588","url":null,"abstract":"Test case selection is a classic testing technique to choose a subset of existing test cases for execution, due to the limited budget and tight deadlines. While 'code coverage' is the state of practice among test case selection heuristics, recent literature has shown that `test case diversity' is also a very promising approach. In this paper, we first compare these two heuristics for test case selection in several real-world case studies (Apache Ant, Derby, JBoss, NanoXML and Math). The results show that neither of the two techniques completely dominates the other, but they can potentially be complementary. Therefore, we next propose a novel approach that maximizes both code coverage and diversity among the selected test cases using NSGA-II multi- objective optimization, and the results show a significant improvement in fault detection rate. Specifically, sometimes this novel approach detects up to 16%(Ant), 10%(JBoss), and 14% (Math) more faults compared to either of coverage or diversity-based approaches, when the testing budget is less than 20% of the entire test suite execution cost.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123080382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Prioritizing Manual Test Cases in Traditional and Rapid Release Environments","authors":"H. Hemmati, Zhihan Fang, M. Mäntylä","doi":"10.1109/ICST.2015.7102602","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102602","url":null,"abstract":"Test case prioritization is one of the most practically useful activities in testing, specially for large scale systems. The goal is ranking the existing test cases in a way that they detect faults as soon as possible, so that any partial execution of the test suite detects maximum number of defects for the given budget. Test prioritization becomes even more important when the test execution is time consuming, e.g., manual system tests vs. automated unit tests. Most existing test case prioritization techniques are based on code coverage, which requires access to source code. However, manual testing is mainly done in a black- box manner (manual testers do not have access to the source code). Therefore, in this paper, we first examine the existing test case prioritization techniques and modify them to be applicable on manual black-box system testing. We specifically study a coverage- based, a diversity-based, and a risk driven approach for test case prioritization. Our empirical study on four older releases of Mozilla Firefox shows that none of the techniques are strongly dominating the others in all releases. However, when we study nine more recent releases of Firefox, where the development has been moved from a traditional to a more agile and rapid release environment, we see a very signifiant difference (on average 65% effectiveness improvement) between the risk-driven approach and its alternatives. Our conclusion, based on one case study of 13 releases of an industrial system, is that test suites in rapid release environments, potentially, can be very effectively prioritized for execution, based on their historical riskiness; whereas the same conclusions do not hold in the traditional software development environments.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"82 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126261220","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Detection and Localization of HTML Presentation Failures Using Computer Vision-Based Techniques","authors":"Sonal Mahajan, William G. J. Halfond","doi":"10.1109/ICST.2015.7102586","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102586","url":null,"abstract":"An attractive and visually appealing appearance is important for the success of a website. Presentation failures in a site's web pages can negatively impact end users' perception of the quality of the site and the services it delivers. Debugging such failures is challenging because testers must visually inspect large web pages and analyze complex interactions among the HTML elements of a page. In this paper we propose a novel automated approach for debugging web page user interfaces. Our approach uses computer vision techniques to detect failures and can then identify HTML elements that are likely to be responsible for the failure. We evaluated our approach on a set of real-world web applications and found that the approach was able to accurately and quickly identify faulty HTML elements.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121755965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"TESEM: A Tool for Verifying Security Design Pattern Applications by Model Testing","authors":"Takanori Kobashi, Masatoshi Yoshizawa, H. Washizaki, Y. Fukazawa, Nobukazu Yoshioka, T. Okubo, H. Kaiya","doi":"10.1109/ICST.2015.7102633","DOIUrl":"https://doi.org/10.1109/ICST.2015.7102633","url":null,"abstract":"Because software developers are not necessarily security experts, identifying potential threats and vulnerabilities in the early stage of the development process (e.g., the requirement- or design-phase) is insufficient. Even if these issues are addressed at an early stage, it does not guarantee that the final software product actually satisfies security requirements. To realize secure designs, we propose extended security patterns, which include requirement-and design-level patterns as well as a new model testing process. Our approach is implemented in a tool called TESEM (Test Driven Secure Modeling Tool), which supports pattern applications by creating a script to execute model testing automatically. During an early development stage, the developer specifies threats and vulnerabilities in the target system, and then TESEM verifies whether the security patterns are properly applied and assesses whether these vulnerabilities are resolved.","PeriodicalId":401414,"journal":{"name":"2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125202975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}