2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)最新文献_第6页

MetaA: Multi-Dimensional Evaluation of Testing Ability via Adversarial Examples in Deep Learning 元:深度学习中基于对抗性示例的测试能力多维评估

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00104

Siqi Gu, Jiawei Liu, Zhan-wei Hui, Wenhong Liu, Zhenyu Chen

{"title":"MetaA: Multi-Dimensional Evaluation of Testing Ability via Adversarial Examples in Deep Learning","authors":"Siqi Gu, Jiawei Liu, Zhan-wei Hui, Wenhong Liu, Zhenyu Chen","doi":"10.1109/QRS57517.2022.00104","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00104","url":null,"abstract":"Deep learning (DL) has shown superior performance in many areas, making the quality assurance of DL-based software particularly important. Adversarial examples are generated by deliberately adding subtle perturbations in input samples and can easily attack less reliable DL models. Most existing works only utilize a single metric to evaluate the generated adversarial examples, such as attacking success rate or structure similarity measure. The problem is that they cannot avoid extreme testing situations and provide multifaceted evaluation results.This paper presents MetaA, a multi-dimensional evaluation framework for testing ability of adversarial examples in deep learning. Evaluating the testing ability represents measuring the testing performance to make improvements. Specifically, MetaA performs comprehensive validation on generating adversarial examples from two horizontal and five vertical dimensions. We design MetaA according to the definition of the adversarial examples and the issue mentioned in [1] that how to enrich the evaluation dimension rather than merely quantifying the improvement of DL and software.We conduct several analyses and comparative experiments vertically and horizontally to evaluate the reliability and effectiveness of MetaA. The experimental results show that MetaA can avoid speculation and reach agreement among different indicators when they reflect inconsistencies. The detailed and comprehensive analysis of evaluation results can further guide the optimization of adversarial examples and the quality assurance of DL-based software.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129165016","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Adopting Misclassification Detection and Outlier Modification to Fault Correction in Deep Learning-Based Systems 基于深度学习系统的误分类检测和离群值修正故障校正

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00108

Chuan-Min Chu, Chin-Yu Huang, Neil C. Fang

{"title":"Adopting Misclassification Detection and Outlier Modification to Fault Correction in Deep Learning-Based Systems","authors":"Chuan-Min Chu, Chin-Yu Huang, Neil C. Fang","doi":"10.1109/QRS57517.2022.00108","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00108","url":null,"abstract":"Over the past few decades, researchers in software engineering (SE) have focused on testing, analyzing, repairing, and generating programs automatically and effectively. Today, combining neural networks and traditional software engineering techniques has major potential to benefit software quality and productivity. Regarding the development of neural networks, deep learning (DL) and convolution neural networks (CNNs) have been widely adopted by software applications for making decisions or providing suggestions. Considering life-critical DL-based applications, there is a need to correct the wrong decisions made by DL systems immediately. Therefore, we propose a novel fault-correction framework for alleviating potential misclassification issues of DL systems called the Outlier Modification for DL Systems (OMDLS). Our experiment results with two public datasets using different scales and label numbers to show that modifying the outliers based on the misclassification pairs can improve accuracy by up to 2.12% without retraining the model and modifying the inference immediately.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128698115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

QRS 2022 Steering Committee QRS 2022指导委员会

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/qrs57517.2022.00006

引用次数: 0

Coverage Testing of Industrial Simulink Models using Monte-Carlo and SMT-Based Methods 基于蒙特卡罗和smt方法的工业Simulink模型覆盖测试

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00050

Daisuke Ishii, Takashi Tomita, Toshiaki Aoki, The Quyen Ngo, Thi Bich Ngoc Do, Hideaki Takai

引用次数: 0

An Effective Low-Dimensional Software Code Representation using BERT and ELMo 基于BERT和ELMo的有效的低维软件代码表示

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00082

Srijoni Majumdar, Ashutosh Varshney, Partha Pratim Das, Paul D. Clough, S. Chattopadhyay

引用次数: 6

Uncertainty-Aware Behavior Modeling and Quantitative Safety Evaluation for Automatic Flight Control Systems 自动飞行控制系统的不确定性感知行为建模与定量安全评估

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00062

Huiyu Liu, Jing Liu, Haiying Sun, Tengfei Li, John Zhang

{"title":"Uncertainty-Aware Behavior Modeling and Quantitative Safety Evaluation for Automatic Flight Control Systems","authors":"Huiyu Liu, Jing Liu, Haiying Sun, Tengfei Li, John Zhang","doi":"10.1109/QRS57517.2022.00062","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00062","url":null,"abstract":"Automatic flight control systems (AFCS) are safety-critical systems tightly integrating computation, networking and physical processes. However, the uncertainty resulting from evolving dynamics in cyberspace and the physical world can affect the reliability of decision-making in the controller, threatening the system’s safety. How to accurately capture the uncertainty, effectively control the aircraft and improve safety has become an unavoidable challenge for the software industry. To this end, we define an uncertainty-aware modeling language (UAML), which supports modeling the AFCS’s dynamic behavior and environmental uncertainty using formal specifications. We use a machine learning-based method to predict the risk levels in operating environments as the representation of uncertainty from the physical world. The prediction result is transferred to UAML as the parameters. On this basis, we present a framework for quantitative safety evaluation using statistical model checking based on UPPAAL-SMC to help AFCS make reliable decisions at runtime. We illustrate our approach by modeling and analyzing a realistic example, and the experimental result demonstrates the effectiveness of our approach.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128030760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An Ontological Analysis of Safety-Critical Software and Its Anomalies 安全关键软件及其异常的本体论分析

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00040

Hezhen Liu, Zhi Jin, Zheng Zheng, Chengqiang Huang, Xun Zhang

{"title":"An Ontological Analysis of Safety-Critical Software and Its Anomalies","authors":"Hezhen Liu, Zhi Jin, Zheng Zheng, Chengqiang Huang, Xun Zhang","doi":"10.1109/QRS57517.2022.00040","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00040","url":null,"abstract":"The progressively dominant role of software in safety-critical systems raise concerns about the software dependability. There are limited mature practices and guides for assessing software dependability and analyzing system-level hazards triggered by software anomalies. A problem is that faults, errors, and failures that represent software anomalies, albeit with different natures, are usually used indistinctly to predict software dependability, leading to unsolid results. The lack of such consensual conceptualization also leads to poor interoperability between supporting tools, and, consequently, difficulties in anomaly management and software maintenance. Anomaly analysis and management is more tough for safety-critical software due to its higher complexity and the safety-critical nature. The complex context of safety-critical software causes difficulties in determining the evolution/propagation path of software anomalies and the impact on system safety. To capture the nature of safety-critical software and support an understanding of mechanisms of software anomalies and associated hazards, we propose three reference ontologies: Safety-critical Software Ontology, Software Fault Ontology and Software-failure-induced Hazard Ontology, which are built based on international standards, guides, and relevant conceptual models. We also discuss the relationships among them. That will facilitate a better understanding of the software anomaly mechanisms and the design of intervening/mitigation solutions. We demonstrate how these ontologies can help analyze software problems of real-world safety-critical systems.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133932252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MC/DC Test Case Automatic Generation for Safety-Critical Systems 安全关键系统的MC/DC测试用例自动生成

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00079

Cong Wang, Haiying Sun, Hui Dou, HongTao Chen, Jing Liu

引用次数: 0

Availability and Cost aware Multi-'omain Service Deployment Optimization 可用性和成本意识多域服务部署优化

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00051

Chuangchuang Zhang, Yanming Liu, Hong-yong Yang, Yihao Li, Shuning Zhang

引用次数: 0

Automated Identification of Performance Changes at Code Level 在代码级别自动识别性能变化

2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2022-12-01 DOI: 10.1109/QRS57517.2022.00096

D. Reichelt, Stefan Kühne, W. Hasselbring

{"title":"Automated Identification of Performance Changes at Code Level","authors":"D. Reichelt, Stefan Kühne, W. Hasselbring","doi":"10.1109/QRS57517.2022.00096","DOIUrl":"https://doi.org/10.1109/QRS57517.2022.00096","url":null,"abstract":"To develop software with optimal performance, even small performance changes need to be identified. Identifying performance changes is challenging since the performance of software is influenced by non-deterministic factors. Therefore, not every performance change is measurable with reasonable effort. In this work, we discuss which performance changes are measurable at code level with reasonable measurement effort and how to identify them. We present (1) an analysis of the boundaries of measuring performance changes, (2) an approach for determining a configuration for reproducible performance change identification, and (3) an evaluation comparing of how well our approach is able to identify performance changes in the application server Jetty compared with the usage of Jetty’s own performance regression benchmarks.Thereby, we find (1) that small performance differences are only measurable by fine-grained measurement workloads, (2) that performance changes caused by the change of one operation can be identified using a unit-test-sized workload definition and a suitable configuration, and (3) that using our approach identifies small performance regressions more efficiently than using Jetty’s performance regression benchmarks.","PeriodicalId":143812,"journal":{"name":"2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122530498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1