2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)最新文献_第3页

More Reliable Test Suites for Dynamic APR by using Counterexamples 通过使用反例为动态APR提供更可靠的测试套件

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00032

Amirfarhad Nilizadeh, Marlon Calvo, Gary T. Leavens, X. Le

{"title":"More Reliable Test Suites for Dynamic APR by using Counterexamples","authors":"Amirfarhad Nilizadeh, Marlon Calvo, Gary T. Leavens, X. Le","doi":"10.1109/ISSRE52982.2021.00032","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00032","url":null,"abstract":"Dynamic automated program repair (APR) techniques, which use test suites for bug localization and evaluating candidate patches, have promising results. However, many studies show that machine-generated patches with dynamic APR tools are not always reliable. Recent studies show that enhancing test suites by adding tests will help dynamic APR tools generate more reliable patches. We evaluate the effectiveness of minimally enhancing test suites by adding counterexamples for repaired programs that suffer from test overfitting. We use formal methods as an independent standard for evaluating patches' correctness and for generating counterexamples. Techniques for evaluating patch correctness (both with human reviewers and formal methods) can create false negatives, meaning that the repaired program is correct but is deemed incorrect. A counterexample is a good way to check on reviewer decisions about correctness. Our study evaluated 256 repaired but not verified programs (from the buggy Java+JML dataset); the repairs were generated by seven state-of-the-art dynamic APR tools. Our results show that the counterexample generated by the OpenJML tool could correctly classify all these programs into the categories of “test overfitting” and “false negatives.” After adding tests based on the counterexamples to the test suites, we ran the APR tools on the original buggy programs again and found that: (1) the APR tools were able to generate about 27.3% more correct patches with the enhanced test suite, and (2) the enhanced test suite resulted in the APR tools generating about 83.6% fewer overfitted patches.","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"163 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133595630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 8

Black-Box and White-Box Test Case Generation for RESTful APIs: Enemies or Allies? RESTful api的黑盒和白盒测试用例生成:敌人还是盟友?

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00034

Alberto Martin-Lopez, Andrea Arcuri, Sergio Segura, Antonio Ruiz-Cortés

{"title":"Black-Box and White-Box Test Case Generation for RESTful APIs: Enemies or Allies?","authors":"Alberto Martin-Lopez, Andrea Arcuri, Sergio Segura, Antonio Ruiz-Cortés","doi":"10.1109/ISSRE52982.2021.00034","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00034","url":null,"abstract":"Automated test case generation for RESTful APIs is a thriving research topic due to their critical role in software integration. Testing approaches can be divided into black-box and white-box. Black-box approaches exploit the API specification for the generation of test cases, while white-box approaches can also leverage the source code. Both strategies have shown great promise, but they have not been fully compared yet, hindering the selection of the right tool for the job. In this paper, we report on our experience comparing black-box and white-box test case generation for RESTful APIs using the state-of-the-art tools RESTest (black-box) and EvoMaster (white-box). Also, we propose integrating both approaches by using black-box test cases as the seed for white-box search-based test case generation. Evaluation results on four RESTful APIs involving over 40 million API calls show that there is no one-size-fits-all strategy. More importantly, the combination of black-box and white- box yielded the best results in most case studies in terms of code coverage and fault finding, paving the way for better tools integrating the best of both perspectives. As a result of our work, we provide lessons learned and open challenges for guiding the use and further development of current tool support.","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133690059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

Organizing Committee ISSRE 2021 组委会ISSRE 2021

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/issre52982.2021.00008

引用次数: 0

Eager Falsification for Accelerating Robustness Verification of Deep Neural Networks 加速深度神经网络鲁棒性验证的急切证伪

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00044

Xingwu Guo, Wenjie Wan, Zhaodi Zhang, Min Zhang, Fu Song, Xuejun Wen

{"title":"Eager Falsification for Accelerating Robustness Verification of Deep Neural Networks","authors":"Xingwu Guo, Wenjie Wan, Zhaodi Zhang, Min Zhang, Fu Song, Xuejun Wen","doi":"10.1109/ISSRE52982.2021.00044","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00044","url":null,"abstract":"Formal robustness verification of deep neural networks (DNNs) is a promising approach for achieving a provable reliability guarantee to AI-enabled software systems. Limited scalability is one of the main obstacles to the verification problem. In this paper, we propose eager falsification to accelerate the robustness verification of DNNs. It divides the verification problem into a set of independent subproblems and solves them in descending order of their falsification probabilities. Once a subproblem is falsified, the verification terminates with a conclusion that the network is not robust. We introduce a notion of label affinity to measure the falsification probability and present an approach to computing the probability based on symbolic interval propagation. Our approach is orthogonal to existing verification techniques. We integrate it into four state-of-the-art verification tools, i.e., MIPVerify, Neurify, DeepZ, and DeepPoly, and conduct extensive experiments on 8 benchmark datasets. The experimental results show that our approach can significantly improve these tools by up to 200x speedup when the perturbation distance is in a reasonable range.","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127841393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

A Characteristic Study of Deadlocks in Database-Backed Web Applications 数据库支持的Web应用中死锁的特性研究

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00059

Zhengyi Qiu, Shudi Shao, Qi Zhao, Guoliang Jin

引用次数: 2

HawkEye: User-Guided Enumeration of Scenarios HawkEye:用户引导的场景枚举

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00064

Allison Sullivan

{"title":"HawkEye: User-Guided Enumeration of Scenarios","authors":"Allison Sullivan","doi":"10.1109/ISSRE52982.2021.00064","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00064","url":null,"abstract":"Writing declarative models has numerous benefits, ranging from automated reasoning and correction of design-level properties before systems are built, to automated testing and debugging of their implementations after they are built. Alloy is a declarative modeling language that is well suited for verifying object-oriented designs. A key strength of Alloy is its scenario-finding toolset the Analyzer, which outputs all valid scenarios that adhere to the model's constraints up to a user-provided scope. However, in order for scenario-finding toolsets to be useful and not an undue burden, scenario-finding toolsets need to generate a relatively small but valuable collection of scenarios. This paper outlines Hawkeye, a novel interactive enumeration technique for the Analyzer that empowers the user to select which elements of a scenario the user wants to keep the same or differ in the next enumeration. Experimental results show that our technique can modify scenario enumeration without significant overhead on the size and complexity of the underlying SAT problem. Moreover, we highlight Hawkeye's ability to help users explore faulty models. Hawkeye is available at: https://github.com/alloy-hawkeye/Hawkeye.git","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131541538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Remembering Dr. Amrit Goel, a Software Reliability Pioneer 记住Amrit Goel博士，软件可靠性先驱

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/issre52982.2021.00005

引用次数: 0

Robust KPI Anomaly Detection for Large-Scale Software Services with Partial Labels 基于部分标签的大型软件服务鲁棒KPI异常检测

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00023

Shenglin Zhang, Chen Zhao, Yicheng Sui, Ya Su, Yongqian Sun, Yuzhi Zhang, Dan Pei, Yizhe Wang

{"title":"Robust KPI Anomaly Detection for Large-Scale Software Services with Partial Labels","authors":"Shenglin Zhang, Chen Zhao, Yicheng Sui, Ya Su, Yongqian Sun, Yuzhi Zhang, Dan Pei, Yizhe Wang","doi":"10.1109/ISSRE52982.2021.00023","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00023","url":null,"abstract":"To ensure the reliability of software services, operators collect and monitor a large number of KPI (Key Performance Indicator) streams constantly. KPI anomaly detection is vitally important for software service management. However, none of supervised learning methods, semi-supervised learning methods, transfer learning methods, or unsupervised learning methods achieve accurate anomaly detection for the large-scale, diverse, dynamically changing KPI streams with little labeling effort. In this paper, we propose PUAD, a PU learning-based method, to achieve accurate KPI anomaly detection requiring a few partial labels. It integrates clustering, PU learning, and semi-supervised learning to minimize labeling effort and improve anomaly detection accuracy simultaneously. Additionally, we propose a novel active learning method that selects the samples most likely to be positive in each iteration to avoid false alarms. We apply 208 real-world KPI streams collected from a large-scale software service provider to evaluate the performance of PUAD, demonstrating that it achieves a close F1-score to supervised learning methods with much fewer manual labels, and greatly outperforms semi-supervised learning methods, transfer learning methods, and unsupervised learning methods.","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123301337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

GCN2defect : Graph Convolutional Networks for SMOTETomek-based Software Defect Prediction GCN2defect:基于smotetome的软件缺陷预测的图卷积网络

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00020

Cheng Zeng, Chunpeng Zhou, Shengkai Lv, Peng He, Jie Huang

{"title":"GCN2defect : Graph Convolutional Networks for SMOTETomek-based Software Defect Prediction","authors":"Cheng Zeng, Chunpeng Zhou, Shengkai Lv, Peng He, Jie Huang","doi":"10.1109/ISSRE52982.2021.00020","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00020","url":null,"abstract":"With the introduction of network metrics into the field of software defect prediction, the dependency network of software modules is widely used. The network embedding models aim to represent nodes as low-dimensional vectors, thereby preserving the topological structure of the network. However, in software engineering, traditional network embedding models do not concern deep learning strategies, while recently, graph neural networks (GNNs) have been proved to be an effective deep learning framework for learning graph data. As a variant of GNN, graph convolution neural network (GCN) has achieved appealing results in node classification and link prediction. Inspired by the performance of GCN, we propose GCN2defect, which extends GCN to automatically learn to encode the software dependency network and ultimately improve software defect prediction. Specifically, we firstly construct a program's Class Dependency Network, and then use node2vec for embedded learning to obtain the structural features of the network automatically. After that, we combine the learned structural features with traditional software code features to initialize the attributes of nodes in the Class Dependency Network. Next, we feed the dependency network to GCN to get much deeper representation of the class. Meanwhile, to enhance the accuracy of prediction, we also employ the SMOTETomek sampling to solve the problem of data imbalance. Finally, we evaluate the proposed method on eight open-source programs and demonstrate that, on average, GCN2defect improves the state-of-the-art approach by 6.84% ~ 23.85% in terms of the F-measure.","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116816515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

A Novel Automatic Query Expansion with Word Embedding for IR-based Bug Localization 一种基于词嵌入的基于ir的Bug定位自动查询扩展方法

2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2021-10-01 DOI: 10.1109/ISSRE52982.2021.00038

Misoo Kim, Youngkyoung Kim, Eunseok Lee

{"title":"A Novel Automatic Query Expansion with Word Embedding for IR-based Bug Localization","authors":"Misoo Kim, Youngkyoung Kim, Eunseok Lee","doi":"10.1109/ISSRE52982.2021.00038","DOIUrl":"https://doi.org/10.1109/ISSRE52982.2021.00038","url":null,"abstract":"Information retrieval-based bug localization (IRBL) aims at finding buggy files using a bug report as a query. IRBL performance is highly dependent on the query quality. To improve the query quality for IRBL, automatic query expansion (AQE) method has been proposed for identifying query-related terms from the first-retrieved source files. This approach inevitably depends on two determinant of post- retrieval results, the retrieval model and the initial query quality. We propose a novel word embedding-based AQE technique, WEQE, to avoid the heavy dependency of the current AQE approach. Word embedding model enables to fetch terms semantically related to a query by representing words in a vector space. Our method embeds the words from both the global corpus and project-specific-corpus. The initial query is extended by adding words semantically similar to it based on vector representations from our embedding model. We validated the effectiveness of WEQE by using 4,583 bug reports from seven projects, four IRBL models, and two em-bedding models. Our large-scale experimental results show that WEQE can improve the average precision for bug localization for at least 42% of all queries. Our expanded queries on the best IRBL model achieve a 6% higher mean average precision for bug localization than the initial query.","PeriodicalId":162410,"journal":{"name":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117282384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4