Matthieu Jimenez, Mike Papadakis, Tegawendé F. Bissyandé, Jacques Klein
{"title":"Profiling Android Vulnerabilities","authors":"Matthieu Jimenez, Mike Papadakis, Tegawendé F. Bissyandé, Jacques Klein","doi":"10.1109/QRS.2016.34","DOIUrl":"https://doi.org/10.1109/QRS.2016.34","url":null,"abstract":"In widely used mobile operating systems a single vulnerability can threaten the security and privacy of billions of users. Therefore, identifying vulnerabilities and fortifying software systems requires constant attention and effort. However, this is costly and it is almost impossible to analyse an entire code base. Thus, it is necessary to prioritize efforts towards the most likely vulnerable areas. A first step in identifying these areas is to profile vulnerabilities based on previously reported ones. To investigate this, we performed a manual analysis of Android vulnerabilities, as reported in the National Vulnerability Database for the period 2008 to 2014. In our analysis, we identified a comprehensive list of issues leading to Android vulnerabilities. We also point out characteristics of the locations where vulnerabilities reside, the complexity of these locations and the complexity to fix the vulnerabilities. To enable future research, we make available all of our data.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124299666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Testing and Debugging in Continuous Integration with Budget Quotas on Test Executions","authors":"Bo Jiang, W. Chan","doi":"10.1109/QRS.2016.66","DOIUrl":"https://doi.org/10.1109/QRS.2016.66","url":null,"abstract":"In Continuous Integration, a software application is developed through a series of development sessions, each with limited time allocated to testing and debugging on each of its modules. Test Case Prioritization can help execute test cases with higher failure estimate earlier in each session. When the testing time is limited, executing such prioritized test cases may only produce partial and prioritized execution coverage data. To identify faulty code, existing Spectrum-Based Fault Localization techniques often use execution coverage data but without the assumption of execution coverage priority. Is it possible to decompose these two steps for optimization within individual steps? In this paper, we study to what extent the selection of test case prioritization techniques may reduce its influence on the effectiveness of spectrum-based fault localization, thereby showing the possibility to decompose the process of continuous integration for optimization in workflow steps. We present a controlled experiment using the Siemens suite as subjects, nine test case prioritization techniques and four spectrum-based fault localization techniques. The findings showed that the studied test cases prioritization and spectrum-based fault localization can be customized separately, and, interestingly, prioritization over a smaller test suite can enable spectrum-based fault localization to achieve higher accuracy by assigning faulty statements with higher ranks.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129917339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Model to Estimate First-Order Mutation Coverage from Higher-Order Mutation Coverage","authors":"Ali Parsai, Alessandro Murgia, S. Demeyer","doi":"10.1109/QRS.2016.48","DOIUrl":"https://doi.org/10.1109/QRS.2016.48","url":null,"abstract":"The test suite is essential for fault detection during software development. First-order mutation coverage is an accurate metric to quantify the quality of the test suite. However, it is computationally expensive. Hence, the adoption of this metric is limited. In this study, we address this issue by proposing a realistic model able to estimate first-order mutation coverage using only higher-order mutation coverage. Our study shows how the estimation evolves along with the order of mutation. We validate the model with an empirical study based on 17 open-source projects.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128791074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Juan Zhai, Bin Li, Zhenhao Tang, Jianhua Zhao, Xuandong Li
{"title":"Precondition Calculation for Loops Iterating over Data Structures","authors":"Juan Zhai, Bin Li, Zhenhao Tang, Jianhua Zhao, Xuandong Li","doi":"10.1109/QRS.2016.25","DOIUrl":"https://doi.org/10.1109/QRS.2016.25","url":null,"abstract":"Precondition calculation is a fundamental program verification technique. Many previous works tried to solve this problem, but ended with limited capability due to loop statements. We conducted a survey on loops manipulating commonly-used data structures occurring in several real-world open-source programs, and found that about 80% of such loops iterate over elements of a data structure, indicating that automatic calculation of preconditions with respect to post-conditions of these loops would cover a great number of real-world programs and greatly ease code verification tasks. In this paper, we specify the execution effect of a program statement using the memories modified by the statement and the new values stored in these memories after executing the statement. Thus, conditional statements and loop statements can be uniformly reduced to a sequence of assignments. Also we present an approach to calculate preconditions with respect to given post-conditions of various program statements including loops that iterate over elements of commonly-used data structures (e.g., acyclic singly-linked lists) based on execution effects of these statements. With execution effects, post-conditions and loop invariants can also be generated. Our approach handles various types of data including numeric, boolean, arrays and user-defined structures. We have implemented the approach and integrated it into the code verification tool, Accumulator. We also evaluated the approach with a variety of programs, and the results show that our approach is able to calculate preconditions for different kinds of post-conditions, including linear ones and universally quantified ones. Preconditions generated with our approach can ease the verification task by reducing the burden of providing loop invariants and preconditions of loop statements manually, which improves the automatic level and efficiency, and makes the verification less error-prone.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129956147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
N. Kushik, Jorge López, A. Cavalli, N. Yevtushenko
{"title":"Improving Protocol Passive Testing through \"Gedanken\" Experiments with Finite State Machines","authors":"N. Kushik, Jorge López, A. Cavalli, N. Yevtushenko","doi":"10.1109/QRS.2016.43","DOIUrl":"https://doi.org/10.1109/QRS.2016.43","url":null,"abstract":"This paper is devoted to study the use of 'gedanken' experiments with Finite State Machines (FSMs) for protocol passive testing optimization. We discuss how the knowledge obtained from the state identification of an implementation under test (IUT) can be utilized for effective IUT monitoring. Differently from active testing techniques, such identification is performed by only observing the IUT behavior. If the state identification is possible (at least partially), then this fact allows to reduce the number of properties (test purposes) to be checked at certain execution point(s). Correspondingly, this allows to simplify and/or accelerate, i.e. improve the monitoring process by verifying the system behavior only at critical states against the appropriate set of properties associated with a given state. The paper discusses which 'gedanken' experiments can be considered for this purpose and how they can be derived for various specifications of communication protocols. The results presented in the paper are followed by an illustrative protocol example that demonstrates the efficiency of the proposed approach.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128290239","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chenglong Sun, Zhenyu Zhang, Bo Jiang, William Chan
{"title":"Facilitating Monkey Test by Detecting Operable Regions in Rendered GUI of Mobile Game Apps","authors":"Chenglong Sun, Zhenyu Zhang, Bo Jiang, William Chan","doi":"10.1109/QRS.2016.41","DOIUrl":"https://doi.org/10.1109/QRS.2016.41","url":null,"abstract":"Graphical User Interface (GUI) is a component of many software applications. Many mobile game applications in particular have to provide excellent user experiences using graphical engines to render GUI screens. On a rendered GUI screen such as a treasury map, no GUI widget is embodied in it and the operable GUI regions, each of which is a region that triggers actions when certain events acting on these regions, may only be implicitly determinable. Traditional testing tools like monkey test do not effectively generate effective event sequences over such operable GUI regions. Our insight is that operable regions in a rendered GUI screen of many mobile game applications are given with visible hints to catch user attentions. In this paper, we propose Smart Monkey, which uses the fundamental features of a screen, including color, intensity, and texture, as visual signals to detect operable GUI region candidates, and iteratively identifies and confirms the real operable GUI regions by launching GUI events to the region. We have implemented Smart Monkey as a testing tool for Android apps and conducted case studies on real-world applications to compare it with a peer technique. The empirical results show that it effective in identifying such operable regions and thus able to generate functional event sequences more efficiently.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125135908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Romain Aïssat, M. Gaudel, Frédéric Voisin, B. Wolff
{"title":"A Method for Pruning Infeasible Paths via Graph Transformations and Symbolic Execution","authors":"Romain Aïssat, M. Gaudel, Frédéric Voisin, B. Wolff","doi":"10.1109/QRS.2016.26","DOIUrl":"https://doi.org/10.1109/QRS.2016.26","url":null,"abstract":"Path-biased random testing is an interesting alternative to classical path-based approaches faced to the explosion of the number of paths, and to the weak structural coverage of random methods based on the input domain only. Given a graph representation of the system under test a probability distribution on paths of a certain length is computed and then used for drawing paths. A limitation of this approach, similarly to other methods based on symbolic execution and static analysis, is the existence of infeasible paths that often leads to a lot of unexploitable drawings. We present a prototype for pruning some infeasible paths, thus eliminating useless drawings. It is based on graph transformations that have been proved to preserve the actual behaviour of the program. It is driven by symbolic execution and heuristics that use detection of subsumptions and the abstract-check-refine paradigm. The approach is illustrated on some detailed examples.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"216 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132525357","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Siyuan Xu, Weikai Miao, T. Kunz, Tongquan Wei, Mingsong Chen
{"title":"Quantitative Analysis of Variation-Aware Internet of Things Designs Using Statistical Model Checking","authors":"Siyuan Xu, Weikai Miao, T. Kunz, Tongquan Wei, Mingsong Chen","doi":"10.1109/QRS.2016.39","DOIUrl":"https://doi.org/10.1109/QRS.2016.39","url":null,"abstract":"Since Internet of Things (IoT) applications are deployed within open physical environments, their executions suffer from a wide spectrum of uncertain factors (e.g., network delay, sensor inputs). Although ThingML is a promising IoT modeling and specification language which enables the fast development of resource-constrained IoT applications, it lacks the capability to model such uncertainties and quantify their effects. Consequently, within uncertain environments the quality and performance of IoT applications generated from ThingML designs cannot be guaranteed. To explore the overall runtime performance variations caused by environmental uncertainties, this paper proposes a quantitative uncertainty evaluation framework for ThingML-based IoT designs. By adopting network of priced timed automata as the model of computation and statistical model checking as the evaluation engine, our approach can model uncertainties caused by external environments as well as support various kinds of performance queries on the extended ThingML designs. Experimental results of two comprehensive case studies demonstrate the efficacy of our approach.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132595234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Srijoni Majumdar, Nachiketa Chatterjee, Shila Rani Sahoo, P. Das
{"title":"D-Cube: Tool for Dynamic Design Discovery from Multi-threaded Applications Using PIN","authors":"Srijoni Majumdar, Nachiketa Chatterjee, Shila Rani Sahoo, P. Das","doi":"10.1109/QRS.2016.13","DOIUrl":"https://doi.org/10.1109/QRS.2016.13","url":null,"abstract":"Program comprehension is a major challenge for system maintenance. Reverse engineering has been employed for control-flow analysis of applications but not much work has been done for comprehending concurrent non-deterministic behavior of multi-threaded applications. We present D-CUBE, built using dynamic instrumentation APIs, which plugs in during execution and infers various thread models like concurrency, safety, data access, thread-pool state, exception model etc. for multi-threaded applications at runtime. We extract run-time events traced according to pre-specified logic and feed them to decision trees for inference. We use 3 benchmark suites (LOC: 50-3200) -- CDAC Pthreads benchmark [1] (18 Cases), Open POSIX Test-Suites [2] (21 Cases) and PARSEC 3.0 benchmarks [3] (3 Cases) for accuracy and volume testing and validate our approach by comparing the documented behavior of test-suites with D-CUBE's output models. We achieve over 90% accuracy. D-CUBE produces graphical event-traces with every inference for quick and effective comprehension of large code.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129530502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Resource/Schedule/Content Test Planning Model","authors":"Pete Rotella","doi":"10.1109/QRS.2016.57","DOIUrl":"https://doi.org/10.1109/QRS.2016.57","url":null,"abstract":"It is necessary to balance the contributions from four primary software testing 'dimensions,' in the integration branch test cycle for waterfall and hybrid waterfall/agile projects, to achieve best-in-class customer experience: 1) Sufficient testing resources (engineers) are needed to ensure that adequate testing is accomplished, 2) sufficient bug fixing resources (engineers) are needed to ensure that the newly uncovered bugs are properly fixed, 3) adequate testing and bug fixing time/schedule are needed to ensure that there is sufficient time to run the test plans completely, to do enough regression testing, and to accommodate dead periods when test stoppers are encountered, and 4) new feature content must not be so high that the testing and fixing teams cannot complete their tasks and produce code that is sufficiently reliable. These four dimensions are key parameters in any testing planning exercise, and often, during the testing cycle itself, one or more of these parameters must be adjusted to satisfy the cost/schedule/reliability goals laid out at the project start. The work described in this paper attempts to construct a generalized model that quantifies the contributions from these dimensions, and enables the practitioner to construct what-if scenarios to dynamically estimate the customers' software reliability experience in their production networks. This Resource/Schedule/ Content Model (RSCM), addressing resources, schedule, and feature content, is based on another useful model, the Universal Release Criteria Model (URC), that gauges the effectiveness of integration-branch testing and fixing, and estimates the software reliability field results. The customer experience metric that is predicted by RSCM and URC is SWDPMH -- software incidents (bug encounters by all customers) per million hours of total fleet usage per month, Cisco's primary measure of reliability in the field.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"107 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123244351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}