2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)最新文献

{"title":"Thresholds for Size and Complexity Metrics: A Case Study from the Perspective of Defect Density","authors":"Kazuhiro Yamashita, Changyun Huang, M. Nagappan, Yasutaka Kamei, A. Mockus, A. Hassan, Naoyasu Ubayashi","doi":"10.1109/QRS.2016.31","DOIUrl":"https://doi.org/10.1109/QRS.2016.31","url":null,"abstract":"Practical guidelines on what code has better quality are in great demand. For example, it is reasonable to expect the most complex code to be buggy. Structuring code into reasonably sized files and classes also appears to be prudent. Many attempts to determine (or declare) risk thresholds for various code metrics have been made. In this paper we want to examine the applicability of such thresholds. Hence, we replicate a recently published technique for calculating metric thresholds to determine high-risk files based on code size (LOC and number of methods), and complexity (cyclomatic complexity and module interface coupling) using a very large set of open and closed source projects written primarily in Java. We relate the threshold-derived risk to (a) the probability that a file would have a defect, and (b) the defect density of the files in the high-risk group. We find that the probability of a file having a defect is higher in the very high-risk group with a few exceptions. This is particularly pronounced when using size thresholds. Surprisingly, the defect density was uniformly lower in the very high-risk group of files. Our results suggest that, as expected, less code is associated with fewer defects. However, the same amount of code in large and complex files was associated with fewer defects than when located in smaller and less complex files. Hence we conclude that risk thresholds for size and complexity metrics have to be used with caution if at all. Our findings have immediate practical implications: the redistribution of Java code into smaller and less complex files may be counterproductive.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134010223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models","authors":"K. E. Bennin, Koji Toda, Yasutaka Kamei, J. Keung, Akito Monden, Naoyasu Ubayashi","doi":"10.1109/QRS.2016.33","DOIUrl":"https://doi.org/10.1109/QRS.2016.33","url":null,"abstract":"To prioritize quality assurance efforts, various fault prediction models have been proposed. However, the best performing fault prediction model is unknown due to three major drawbacks: (1) comparison of few fault prediction models considering small number of data sets, (2) use of evaluation measures that ignore testing efforts and (3) use of n-fold cross-validation instead of the more practical cross-release validation. To address these concerns, we conducted cross-release evaluation of 11 fault density prediction models using data sets collected from 2 releases of 25 open source software projects with an effort-aware performance measure known as Norm(Popt). Our result shows that, whilst M5 and K* had the best performances, they were greatly influenced by the percentage of faulty modules present and size of data set. Using Norm(Popt) produced an overall average performance of more than 50% across all the selected models clearly indicating the importance of considering testing efforts in building fault-prone prediction models.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129018400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"DFL: Dual-Service Fault Localization","authors":"C. M. Tang, J. Keung, Yuen-Tak Yu, W. Chan","doi":"10.1109/QRS.2016.53","DOIUrl":"https://doi.org/10.1109/QRS.2016.53","url":null,"abstract":"In engineering a service, software developers often construct and deploy a newer (forthcoming) version of the service to replace the current version. A forthcoming version is often placed online for users to consume and report feedback. In the case of observed failures, the forthcoming version should be debugged and further evolved. In this paper, we propose the model of dual-service fault localization (DFL) to aid this evolution process. Many prior research studies on spectrum-based fault localization (SBFL) consider each version separately. The DFL model correlates the dynamic execution spectra of the current and the forthcoming versions of the same service placed for live test of the forthcoming version, and dynamically generates an adaptive fault localization formula to estimate the code regions in the forthcoming service responsible for the observed failures. We report an experiment in which we initialized the DFL model into six instances, each using an ensemble technique dynamically composed from 11 existing SBFL formulas, and applied the model to four benchmarks. The results show that DFL is feasible and multiple instances are statistically more effective than, if not as effective as, the best of these individual SBFL formulas on each benchmark.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127537995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Protecting Legacy Code against Control Hijacking via Execution Location Equivalence Checking","authors":"Tobias F. Pfeffer, Stefan Sydow, Joachim Fellmuth, Paula Herber","doi":"10.1109/QRS.2016.35","DOIUrl":"https://doi.org/10.1109/QRS.2016.35","url":null,"abstract":"Current anomaly detection systems that enforce control flow integrity based on control flow graph information are not able to precisely monitor dynamic aspects of execution. Consequently, they are typically too coarse-grained to comprehensively detect modern code-reuse attacks. Even when enriched with dynamic monitoring information such as shadow stacks, the heuristics used are either too imprecise or produce many false negatives. In this paper, we present a novel approach to establish control flow integrity in multi-variant execution through execution location equivalence. The concept of execution location equivalence allows us to precisely detect execution divergence using a diversified control flow model and, consequently, to detect a broad variety of code-reuse attacks. In this way, execution of position-independent executables can be reliably rotected against a broad range of control hijacking attacks.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125206594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Leveraging Static Analysis Tools for Improving Usability of Memory Error Sanitization Compilers","authors":"Rigel Gjomemo, Phu H. Phung, Edmund Ballou, Kedar S. Namjoshi, V. Venkatakrishnan, L. Zuck","doi":"10.1109/QRS.2016.44","DOIUrl":"https://doi.org/10.1109/QRS.2016.44","url":null,"abstract":"Memory errors such as buffer overruns are notorious security vulnerabilities. There has been considerable interest in having a compiler to ensure the safety of compiled code either through static verification or through instrumented runtime checks. While certifying compilation has shown much promise, it has not been practical, leaving code instrumentation as the next best strategy for compilation. We term such compilers Memory Error Sanitization Compilers (MESCs). MESCs are available as part of GCC, LLVM and MSVC suites. Due to practical limitations, MESCs typically apply instrumentation indiscriminately to every memory access, and are consequently prohibitively expensive and practical to only small code bases. This work proposes a methodology that applies state-of-the-art static analysis techniques to eliminate unnecessary runtime checks, resulting in more efficient and scalable defenses. The methodology was implemented on LLVM's Safecode, Integer Overflow, and Address Sanitizer passes, using static analysis of Frama-C and Codesurfer. The benchmarks demonstrate an improvement in runtime performance that makes incorporation of runtime checks a viable option for defenses.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117010006","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"How Does the Dependence between Inspectors Impact Capture-Recapture Estimation? An Empirical Study","authors":"Ou Wu","doi":"10.1109/QRS.2016.51","DOIUrl":"https://doi.org/10.1109/QRS.2016.51","url":null,"abstract":"Studies on Capture-recapture (CRC) method in software engineering suggested that the accuracy of the estimates were still far from satisfying. Although debates exist, researchers believe the dependence between different inspectors is one of the factors impacting the accuracy of CRC estimates. In this study, we apply linear correlation to measure the degree of dependence (measured by correlation coefficient, i.e., COV) between inspectors, based on which, we investigate the relation between COV and estimation accuracy (measure by Relative Error, i.e., RE) using the Unconditional Maximum Likelihood Estimator (Mt-UMLE) through nine sets of data from five papers. Besides, the confidence interval (CI)-(which is also an important indicator to reflect CRC estimating accuracy), may also be impacted by COV, to this end, we also investigated the relation between COV and CI. The major results show that: a) It is clear that there is an obviously inverse proportion on COV and CI, and when the COV=0, the radius of CI is almost in interval [1, 6] (exclude the outliers). b) There has also an inverse proportion with COV and RE, and the absolute value of RE increased with the absolute value of COV increased. So we can conclude that the accuracy of CRC estimates had been impacted by the dependence of inspectors. Our study suggested that the correlation cannot be ignored in the study of CRC.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129249909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards Understanding Interactive Debugging","authors":"Fábio Petrillo, Z. Soh, Foutse Khomh, M. Pimenta, C. Freitas, Yann-Gaël Guéhéneuc","doi":"10.1109/QRS.2016.27","DOIUrl":"https://doi.org/10.1109/QRS.2016.27","url":null,"abstract":"Debugging is a laborious activity in which developers spend lot of time navigating through code, looking for starting points, and stepping through statements. Yet, although debuggers exist for 40 years now, there have been few research studies to understand this important and laborious activity. Indeed, to perform such a study, researchers need detailed information about the different steps of the interactive debugging process. In this paper, to help research studies on debugging and, thus, help improving our understanding of how developers debug systems using debuggers, we present the Swarm Debug Infrastructure (SDI), with which practitioners and researchers can collect and share data about developers' interactive debugging activities. We assess the effectiveness of the SDI through an experiment that aims to understand how developers apply interactive debugging on five true faults found in JabRef, toggling breakpoints and stepping code. Our study involved five freelancers and two student developers performing 19 bug location sessions. We collect videos recording and data about 6 hours of effective debugging activities. The data includes 110 breakpoints and near 7,000 invocations. We process the collected videos and data to answer five research questions showing that (1) there is no correlation between the number of invocations (respectively the number of breakpoints toggled) during a debugging session and the time spent on the debugging task, ρ = -0.039 (respectively 0.093). We also observed that (2) developers follow different debugging patterns and (3) there is no relation between numbers of breakpoints and expertise. However, (4) there is a strong negative correlation between time of the first breakpoint (ρ = -0.637), and the time spent on the task, suggesting that when developers toggle breakpoints carefully, they complete tasks faster than developers who toggle breakpoints too quickly. We conclude that the SDI allows collecting and sharing debugging data that can provide interesting insights about interactive debugging activities. We discuss some implications for tool developers and future debuggers.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"94 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121492499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Bugs Framework (BF): A Structured Approach to Express Bugs","authors":"Irena Bojanova, P. Black, Y. Yesha, Yan Wu","doi":"10.1109/QRS.2016.29","DOIUrl":"https://doi.org/10.1109/QRS.2016.29","url":null,"abstract":"To achieve higher levels of assurance for digital systems, we need to answer questions such as does this software have bugs of these critical classes? Do two software assurance tools find the same set of bugs or different, complimentary sets? Can we guarantee that a new technique discovers all problems of this type? To answer such questions, we need a vastly improved way to describe classes of vulnerabilities and chains of failures. We present the Bugs Framework (BF), which raises the current realm of best efforts and useful heuristics. Our BF includes rigorous definitions and (static) attributes of bug classes, along with their related dynamic properties, such as proximate, secondary and tertiary causes, consequences and sites. The paper discusses the buffer overflow class, the injection class and the control of interaction frequency class, and provides examples of applying our BF taxonomy to describe particular vulnerabilities.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"178 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134237094","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automatic Invariant Synthesis for Arrays in Simple Programs","authors":"Bin Li, Zhenhao Tang, Juan Zhai, Jianhua Zhao","doi":"10.1109/QRS.2016.23","DOIUrl":"https://doi.org/10.1109/QRS.2016.23","url":null,"abstract":"This paper proposes a way of using abstract interpretation for discovering properties about array contents in programs which manipulate arrays by sequential traversal. The method summarizes an array property as a universally quantified property. It directly treats invariant properties (including universally quantified formulas and atomic formulas) as abstract domains. Our method is sound and converges in finite time, and it is flexible. The method has been used to automatically discover nontrivial invariants for several examples. In particular, the method can represent and process multidimensional array properties.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131495994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Model-Driven Evaluation of Software Architecture Quality Using Model Clone Detection","authors":"Matthew Stephan, J. Cordy","doi":"10.1109/QRS.2016.21","DOIUrl":"https://doi.org/10.1109/QRS.2016.21","url":null,"abstract":"As software architecture methods and tools become increasingly model-driven, evaluating architecture artifacts must adjust correspondingly. Model-driven evaluation of architecture quality has advantages over traditional evaluation techniques, especially when applied in a model-driven context. One approach we found successful in performing model-driven analysis involves using model clone detection, whereby we detect subsystems that are similar to example systems that are positive and negative quality indicators. In this paper we present our ideas on applying model clone detection to realize model-driven evaluation of software architectures, which contain many high-level systems and interactions. We propose having model-based representations of architectural patterns and styles, and employing model clone detection to identify positive and negative architectural aspects for evaluation, including reliability and security. We provide our insights on how this research can be applied to popular architectural paradigms, relation to previous work, and present discussion points on how it will impact software architecture quality evaluation.","PeriodicalId":412973,"journal":{"name":"2016 IEEE International Conference on Software Quality, Reliability and Security (QRS)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132689174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}