{"title":"Learning and Adaptive Testing of Nondeterministic State Machines","authors":"A. Petrenko, Florent Avellaneda","doi":"10.1109/QRS.2019.00053","DOIUrl":"https://doi.org/10.1109/QRS.2019.00053","url":null,"abstract":"The paper addresses the problems of active learning and conformance testing of systems modeled by nondeterministic Mealy machines (NFSM). It presents a unified SAT-based approach originally proposed by the authors for deterministic FSMs and now generalized to partial nondeterministic machines and checking experiments. Learning a nondeterministic black box, the approach neither needs a Teacher nor uses it a conformance tester to approximate equivalence queries. The idea behind this approach is to infer from a current set of traces not one, but two inequivalent conjectures, use an input sequence distinguishing them in an output query, and update the current trace set with an observed trace to obtain a new pair of distinguishable conjectures, if possible. The classical active learning problem is further generalized by adding a nondeterministic specification FSM, which defines the solution space. The setup unifies the learning and adaptive testing problems and makes them equisolvable with the proposed approach.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123817588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Srijoni Majumdar, Shakti Papdeja, P. Das, S. Ghosh
{"title":"SMARTKT: A Search Framework to Assist Program Comprehension using Smart Knowledge Transfer","authors":"Srijoni Majumdar, Shakti Papdeja, P. Das, S. Ghosh","doi":"10.1109/QRS.2019.00026","DOIUrl":"https://doi.org/10.1109/QRS.2019.00026","url":null,"abstract":"Regardless of attempts to extract knowledge from code bases to aid in program comprehension, there is an absence of a framework to extract and integrate knowledge to provide a near-complete multifaceted understanding of a program. To bridge this gap, we propose SMARTKT (Smart Knowledge Transfer) to extract and transfer knowledge related to software development and application-specific characteristics and their interrelationships in form of a knowledge graph. For an application, the knowledge graph provides an overall understanding of the design and implementation and can be used by an intelligent natural language query system to convert the process of knowledge transfer into a developer-friendly Google-like search. For validation, we develop an analyzer to discover concurrency-related design aspects from runtime traces in a machine learning framework and obtain a precision and recall of around 97% and 95% respectively. We extract application-specific knowledge from code comments and obtain 72% match against human-annotated ground truth.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130573772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zachary B. Ratliff, D. Richard Kuhn, Daniel J. Ragsdale
{"title":"Detecting Vulnerabilities in Android Applications using Event Sequences","authors":"Zachary B. Ratliff, D. Richard Kuhn, Daniel J. Ragsdale","doi":"10.1109/QRS.2019.00032","DOIUrl":"https://doi.org/10.1109/QRS.2019.00032","url":null,"abstract":"Sequence covering arrays have demonstrated their usefulness for finding software bugs that propagate via some sequence of events. However, the distribution of t-way event sequence failures has never been reported, and as a result, the practicality of using these methods is not fully known. In this paper, our analysis of the distribution of t-way interactions between events in event sequence bugs provides insight into the practicality and usefulness of this combinatorial testing method. From a developer's perspective, these methods can contribute to finding this particular class of bugs early in the software development process, saving the developers time and money without sacrificing effectiveness. However, an attacker may also leverage these techniques to discover previously undetected vulnerabilities as a means to exploit the system. This work involved analyzing hundreds of vulnerability reports, performing event sequence testing on two different closed source Android applications, as well as developing a combinatorial coverage measurement tool.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134391454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluating Fault Localization for Resource Adaptation via Test-Based Software Modification","authors":"Arpit Christi, Alex Groce, Rahul Gopinath","doi":"10.1109/QRS.2019.00017","DOIUrl":"https://doi.org/10.1109/QRS.2019.00017","url":null,"abstract":"The ability to dynamically adapt to resource variations is critical for modern-day mission-critical systems that operate in ever-changing resource environments. Test-based Software Modification (TBSM) is a recently proposed technique to build Resource Adaptive Software (RAS) that relies on existing test infrastructure, test labeling, and program modifications. TBSM is simple and applicable, but an inefficient technique; the primary reason for inefficiency is the sheer size of the search space. In this paper, we propose AdFL, a repurposing of Fault Localization (FL) that can shrink (and prioritize) the search space for TBSM more effectively than previously proposed heuristics. We present complete case studies and an empirical analysis of a set of open source projects as evidence that AdFL can significantly reduce the search space in TBSM. We show how to combine AdFL with previous heuristics for TBSM, and propose an incremental, best-effort variant of TBSM that uses AdFL to prioritize the search.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124953164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Technical Debt Prioritization: A Search-Based Approach","authors":"Reem Alfayez, B. Boehm","doi":"10.1109/QRS.2019.00060","DOIUrl":"https://doi.org/10.1109/QRS.2019.00060","url":null,"abstract":"Technical Debt (TD) prioritization is the process of deciding which TD items should be repaid first and which items can be endured until later releases. The goal of the process is to maximize the value of the TD repayment with limited resources. Unfortunately, researchers have indicated the scarcity of TD prioritization techniques and limitations in them. To address these limitations, we propose a novel search-based approach for prioritizing TD using a Multi-objective Evolutionary Algorithm (MOEA). The approach indicates which TD items should be repaid to maximize the value of a repayment activity within a specific cost constraint. An empirical evaluation that we performed on 40 Open-Source Software (OSS) systems demonstrated our approach's ability to improve the value of TD repayment by 1,796 over random search. Additionally, a user study that we conducted with developers confirmed the suitability of our approach in industry and its usefulness in improving the value of TD repayment over developers' prioritization by 423.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"54 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123210459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Florian Klück, Martin Zimmermann, F. Wotawa, M. Nica
{"title":"Genetic Algorithm-Based Test Parameter Optimization for ADAS System Testing","authors":"Florian Klück, Martin Zimmermann, F. Wotawa, M. Nica","doi":"10.1109/QRS.2019.00058","DOIUrl":"https://doi.org/10.1109/QRS.2019.00058","url":null,"abstract":"In this paper, we outline the use of a genetic algorithm for test parameter optimization in the context of autonomous and automated driving. Our approach iteratively optimizes test parameters to aim at obtaining critical scenarios that form the basis for virtual verification and validation of Advanced Driver Assistant Systems (ADAS). We consider a test scenario to be critical if the underlying parameter set causes a malfunction of the system equipped with the ADAS function (i.e., near crash or crash of the vehicle). For evaluating the effectiveness of our approach, we set up an automated simulation framework, where we simulated the Euro NCAP car-to-car rear scenario. To assess the criticality of each test scenario we rely on time-to-collision (TTC), which is a well-known and often used time-based safety indicator for recognizing rear-end conflicts. Our genetic algorithm approach showed a higher chance to generate a critical scenario, compared to a random selection of test parameters.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115206224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Characterizing Software Maintainability in Issue Summaries using a Fuzzy Classifier","authors":"Celia Chen, Michael Shoga, B. Boehm","doi":"10.1109/QRS.2019.00029","DOIUrl":"https://doi.org/10.1109/QRS.2019.00029","url":null,"abstract":"Despite the importance of software maintainability in the life cycle of software systems, accurate measurement remains difficult to achieve. Previous work has shown how bug reports can be classified by expressed quality concerns which can give insight into maintainability across domains and over time. However, the amount of manual effort required to produce such classifications limits its usage. In this paper, we build a fuzzy classifier with linguistic patterns to automatically map issue summaries into the seven subgroup SQ classifications provided in a software maintainability ontology. We investigate how long it takes to generate a stable set of rules and evaluate the performance of the rule set on both rule generating and nonrule generating projects. The results validate the generalizability of the fuzzy classifier in correctly and automatically identifying the subgroup SQ classifications from given issue summaries. This provides a building block for analyzing project maintainability on a larger scale.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131010304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"When and Why Do Software Developers Face Uncertainty?","authors":"Naoyasu Ubayashi, Yasutaka Kamei, Ryosuke Sato","doi":"10.1109/QRS.2019.00045","DOIUrl":"https://doi.org/10.1109/QRS.2019.00045","url":null,"abstract":"Recently, many developers begin to notice that uncertainty is a crucial problem in software development. Unfortunately, no one knows how often uncertainty appears or what kinds of uncertainty exist in actual projects, because there are no empirical studies on uncertainty. To deal with this problem, we conduct a large-scale empirical study analyzing commit messages and revision histories of 1,444 OSS projects randomly selected from the GitHub repositories. The main findings are as follows: 1) Uncertainty exists in the ratio of 1.44% (average); 2) Uncertain program behavior, uncertain variable/value/name, and uncertain program defects are major kinds of uncertainty; and 3) Sometimes developers tend to take an action for not resolving but escaping or ignoring uncertainty. Uncertainty exists everywhere in a certain percentage and developers cannot ignore the existence of uncertainty.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126669576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"First International Workshop on Data-driven System Quality Assurance Workshop Summary","authors":"B. Boehm, Xavier Franch","doi":"10.1109/qrs.2019.00013","DOIUrl":"https://doi.org/10.1109/qrs.2019.00013","url":null,"abstract":"The QRS19 conference hosted a full-day workshop on the data-driven system quality assurance. The workshop, with acronym DSQA, offered a keynote, six presentations, a tutorial and a final discussion session on the topic.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"182 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121827270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Tool-Supported Analysis of Dynamic and Stochastic Behaviors in Cyber-Physical Systems","authors":"Li Huang, Tian Liang, E. Kang","doi":"10.1109/QRS.2019.00039","DOIUrl":"https://doi.org/10.1109/QRS.2019.00039","url":null,"abstract":"Formal analysis of functional and non-functional requirements is crucial in cyber-physical systems (CPS), in which controllers interact with physical environments. The continuous time behaviors of CPS often rely on complex dynamics as well as on stochastic behaviors. We have previously proposed a probabilistic extension of Clock Constraint Specification Language, called PrCCSL, for specification of (non)-functional requirements of CPS and proved the correctness of requirements by mapping the semantics of the specifications into verifiable UPPAAL models. Previous work is extended in this paper by including an extension of PrCCSL, i.e., PrCCSL*, which incorporates annotations of continuous behaviors and stochastic characteristics of CPS. The CPS behaviors are specified in PrCCSL* and translated into stochastic UPPAAL models for formal verification. The translation algorithm from PrCCSL* into UPPAAL models is provided and implemented in an automatic translation tool, namely ProTL. Formal verification of CPS against (non)-functional requirements is performed by ProTL using UPPAAL-SMC as an analysis backend. Our approach is demonstrated on a series of CPS case studies.","PeriodicalId":122665,"journal":{"name":"2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123803106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}