{"title":"Source-Codeless Testing for Android Apps","authors":"Camilo Escobar-Velásquez","doi":"10.1109/icst46399.2020.00057","DOIUrl":"https://doi.org/10.1109/icst46399.2020.00057","url":null,"abstract":"Most of the automated approaches to support software engineering tasks for mobile applications, use source code as entry, which due to privacy factors imposes hard constraints on the implementation of those approaches by third-party services. However, the continuous pressure from market is leading practitioners to crowdsource/outsource software engineering tasks to third-parties that provide on-the-cloud infrastructures. Therefore, approaches that work at closed-source level (i.e., do not require source code) are desirable to enable automated outsourced software engineering tasks by third-party services. Specifically, in this paper we focus on testing since is a task that has broadly accepted approaches at source-code level, but new efforts are required to enable source-codeless approaches. We analyzed the current challenges and shortfalls of the currently proposed approaches that works at source code and APK level, and present our research plan oriented to reduce the existing gap for enabling testing techniques at APK level.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125458494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Substate Profiling for Enhanced Fault Detection and Localization: An Empirical Study","authors":"R. A. Assi, Wes Masri, Chadi Trad","doi":"10.1109/ICST46399.2020.00013","DOIUrl":"https://doi.org/10.1109/ICST46399.2020.00013","url":null,"abstract":"Researchers have used execution profiles to enable coverage-based techniques in areas such as defect detection and fault localization. Typical profile elements include functions, statements, and branches, which are structural in nature. Such elements might not always discriminate failing runs from passing runs, which renders them ineffective in some cases. This motivated us to investigate alternative profiles, namely, substate profiles that aim at approximating the state of a program (as opposed to its execution path). Substate profiling is a recently presented form of state profiling that is practical, fine-grained, and generic enough to be applicable to various profile-based analyses.This paper presents an empirical study demonstrating how complementing structural profiles with substate profiles would benefit Test Suite Reduction (TSR), Test Case Prioritization (TCP), and Spectrum-based Fault Localization (SBFL). Using the Defects4J benchmark, we contrasted the effectiveness of TSR, TCP, and SBFL when using the structural profiles only to when using the concatenation of the structural and substate profiles. Leveraging substate profiling enhanced the effectiveness of all three techniques. For example: 1) For TSR, 86 more versions exhibited 100% defect detection rate. 2) For TCP, 22 more versions had one of their failing tests ranked among the top 20%. 3) For SBFL,substate profiling localized 14 faults that structural profiling failed to localize. Furthermore, our study showed that the improvement due tosubstate profiling was noticeably more significant in the presence of coincidentally correct tests than in their absence. This positions substate profiling as a promising basis for mitigating the negative effect of coincidental correctness.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"2004 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125789858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Supriya Agrawal, R. Venkatesh, U. Shrotri, Amey Zare, S. Verma
{"title":"Scaling Test Case Generation For Expressive Decision Tables","authors":"Supriya Agrawal, R. Venkatesh, U. Shrotri, Amey Zare, S. Verma","doi":"10.1109/icst46399.2020.00044","DOIUrl":"https://doi.org/10.1109/icst46399.2020.00044","url":null,"abstract":"Conventional automated test case generation techniques do not scale to modern software systems, as these systems have a large number of requirements that change frequently. In this paper, we present a scalable algorithm, AGenT, that generates test cases to cover maximal requirements.AGenT takes Expressive Decision Tables (EDT), specifying requirements of a system, as input and realises these as multiple Discrete Time Automata (DTAs). AGenT then generates test cases to cover each row of the tables. To improve scalability, it attempts to cover nearer rows (requiring fewer inputs) first, where distance is measured using a novel distance-to-match heuristic. It also maintains information about desirability and predictability of inputs so as to select promising inputs with a higher probability. Although the algorithm has been presented in the context of EDT, it operates on its DTA representation and hence can be applied to any system that is represented as a collection of DTAs like Statemate and Stateflow. In this paper, we describe AGenT in detail and present findings from two experiments that we conducted. We compared AGenT with state-of-the-art algorithms, DRAFT and a random test case generation algorithm, RTG. In the first experiment, AGenT took a maximum of 144 seconds to cover all rows whereas the other two algorithms timed out on many modules. In the second experiment, for a module with 701 rows, AGenT achieved 7% more coverage than DRAFT and 12% more than RTG.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126827290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Élodie Bernard, Julien Botella, F. Ambert, B. Legeard, M. Utting
{"title":"Tool Support for Refactoring Manual Tests","authors":"Élodie Bernard, Julien Botella, F. Ambert, B. Legeard, M. Utting","doi":"10.1109/icst46399.2020.00041","DOIUrl":"https://doi.org/10.1109/icst46399.2020.00041","url":null,"abstract":"Manual test suites are typically described by natural language, and over time large manual test suites become disordered and harder to use and maintain. This paper focuses on the challenge of providing tool support for refactoring such test suites to make them more usable and maintainable. We describe how we have applied various machine-learning and NLP techniques and other algorithms to the refactoring of manual test suites, plus the tool support we have built to embody these techniques and to allow test suites to be explored and visualised. We evaluate our approach on several industry test suites, and report on the time savings that were obtained.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129890908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Marco Pistoia, Peng Liu, Chun Chen, Shaohan Hu, Stephen P Wood
{"title":"Parallelization of Classical Numerical optimization in Quantum Variational Algorithms","authors":"Marco Pistoia, Peng Liu, Chun Chen, Shaohan Hu, Stephen P Wood","doi":"10.1109/ICST46399.2020.00039","DOIUrl":"https://doi.org/10.1109/ICST46399.2020.00039","url":null,"abstract":"Numerical optimization has been extensively used in many real-world applications related to Scientific Computing, Artificial Intelligence and, more recently, Quantum Computing. However, existing optimizers conduct their internal computations sequentially, which affects their performance. We observed a general pattern that enabled us to parallelize such internal computations and achieve significant speedup. We designed a novel parallelization algorithm for optimizers, which consists of pattern detection, prediction, precomputation, and caching. Importantly, our design does not require any change to the optimizers. Instead, it simply modifies the function to be optimized, thereby leading to several engineering advantages, including simplicity, modularity and portability. We implemented this solution and included it in the Qiskit Aqua open-source project. In this paper, we present an evaluation on both standard benchmarks and real-world quantum-computing applications. The evaluation results confirm that our approach (1) incurs negligible overhead, (2) effectively speeds up optimization, and (3) does not affect the accuracy of the results or the convergence of the optimizers.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130900878","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Guimarães, Leo Fernandes, Márcio Ribeiro, Marcelo d’Amorim, Rohit Gheyi
{"title":"Optimizing Mutation Testing by Discovering Dynamic Mutant Subsumption Relations","authors":"M. Guimarães, Leo Fernandes, Márcio Ribeiro, Marcelo d’Amorim, Rohit Gheyi","doi":"10.1109/icst46399.2020.00029","DOIUrl":"https://doi.org/10.1109/icst46399.2020.00029","url":null,"abstract":"One recent promising direction on reducing costs of mutation analysis is to identify redundant mutations, i.e., mutations that are subsumed by some other mutations. Previous works found out redundant mutants manually through the truth table. Although the idea is promising, it can only be applied for logical and relational operators. In this paper, we propose an approach to discover redundancy in mutations through dynamic subsumption relations among mutants. We focus on subsumption relations among mutations of an expression or statement, named here as “mutation target:” By focusing on targets and relying on automatic test generation tools, we define subsumption relations for dozens of mutation targets in which the MUJAVA tool can apply mutations. We then implemented these relations in a tool, named MUJAVA-M, that generates a reduced set of mutants for each target, avoiding redundant mutants. We evaluated MUJAVA and MUJAVA-M using classes of five open-source projects. As results, we analyze 2,341 occurrences of 32 mutation targets in 168 classes. MUJAVA-M generates less mutants (on average 64.43% less) with 100% of effectiveness in 20 out of 32 targets and more than 95% in 29 out of 32 mutation targets. MUJAVA- M also reduced the time to execute the test suites against the mutants in 52.53% on average, considering the full mutation analysis process.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133889537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"One-click formal methods","authors":"Liana Hadarean","doi":"10.1109/icst46399.2020.00009","DOIUrl":"https://doi.org/10.1109/icst46399.2020.00009","url":null,"abstract":"Formal methods have been successfully applied in domains such as microprocessor hardware design and aerospace, e.g., proofs of security properties for helicopter control systems [4]. However, despite 50 years of research and development, we have not seen wide adoption of formal methods for large and complex systems such as web services, industrial automation, or enterprise support software. One of the key difficulties when proving security, safety, and robustness of these systems is the problem of finding the models of system architectures necessary for analysis. Additionally, the size of the potential user community and the business value typically does not justify the creation of scalable and easy-to-use tools for formal verification.","PeriodicalId":235967,"journal":{"name":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114502084","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}