2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)最新文献_第4页

Fault Injection to Generate Failure Data for Failure Prediction: A Case Study 故障注入生成故障数据用于故障预测:一个案例研究

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00020

João R. Campos, E. Costa

{"title":"Fault Injection to Generate Failure Data for Failure Prediction: A Case Study","authors":"João R. Campos, E. Costa","doi":"10.1109/ISSRE5003.2020.00020","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00020","url":null,"abstract":"Due to the complexity of modern software, identifying every fault before deployment is extremely difficult or even not possible. Such residual faults can ultimately lead to failures, often incurring considerable risks or costs. Online Failure Prediction (OFP) is a fault-tolerance technique that attempts to predict the occurrence of failures in the near future and thus prevent/mitigate their consequences. Combined with recent technological developments, Machine Learning (ML) has been successfully used to create predictive models for OFP. However, as failures are rare events, failure data are often not available for building accurate models. Although fault injection has been accepted as a viable solution to generate realistic failure data, fault injectors are difficult to implement/update and thus research on Operating System (OS)-level OFP has become stale, with most works using data from outdated OSs. In this paper, we conduct a comprehensive fault injection campaign on an up-to-date Linux kernel and thoroughly study its behavior in the presence of faults. We then transform the data to explore and assess the predictive performance of various ML techniques for OFP. Finally, we study the influence of different OFP parameters (i.e., lead-time, prediction-window) and compare the results with existing related work. Results suggest that the various failures observed can be grouped into categories that can then be accurately predicted and distinguished by diverse ML models.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127261871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Unsupervised Detection of Microservice Trace Anomalies through Service-Level Deep Bayesian Networks 基于服务级深度贝叶斯网络的微服务跟踪异常无监督检测

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00014

Ping Liu, Haowen Xu, Qianyu Ouyang, Rui Jiao, Zhekang Chen, Shenglin Zhang, Jiahai Yang, Linlin Mo, Jice Zeng, Wenman Xue, Dan Pei

{"title":"Unsupervised Detection of Microservice Trace Anomalies through Service-Level Deep Bayesian Networks","authors":"Ping Liu, Haowen Xu, Qianyu Ouyang, Rui Jiao, Zhekang Chen, Shenglin Zhang, Jiahai Yang, Linlin Mo, Jice Zeng, Wenman Xue, Dan Pei","doi":"10.1109/ISSRE5003.2020.00014","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00014","url":null,"abstract":"The anomalies of microservice invocation traces (traces) often indicate that the quality of the microservice-based large software service is being impaired. However, timely and accurately detecting trace anomalies is very challenging due to: 1) the large number of underlying microservices, 2) the complex call relationships between them, 3) the interdependency between the response times and invocation paths. Our core idea is to use machine learning to automatically learn the overall normal patterns of traces during periodic offline training. In online anomaly detection, a new trace with a small anomaly score (computed based on the learned normal pattern) is considered anomalous. With our novel trace representation and the design of deep Bayesian networks with posterior flow, our unsupervised anomaly detection system, called TraceAnomaly, can accurately and robustly detect trace anomalies in a unified fashion. TraceAnomaly has been deployed on 18 online services in a company S. Detailed evaluations on four large online services which contain hundreds of microservices and a testbed which contains 41 microservices show that the recall and precision of TraceAnomaly are both above 0.97, outperforming the existing approach in S (hard-coded rule) by 19.6% and 7.1%, and seven other baselines by 57.0% and 41.6% on average.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124899591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 56

QoS-aware Metamorphic Testing: An Elevation Case Study 质量感知的变形测试:一个高程案例研究

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00019

J. Ayerdi, Sergio Segura, Aitor Arrieta, Goiuria Sagardui Mendieta, Maite Arratibel

{"title":"QoS-aware Metamorphic Testing: An Elevation Case Study","authors":"J. Ayerdi, Sergio Segura, Aitor Arrieta, Goiuria Sagardui Mendieta, Maite Arratibel","doi":"10.1109/ISSRE5003.2020.00019","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00019","url":null,"abstract":"Elevators are among the oldest and most widespread transportation systems, yet their complexity increases rapidly to satisfy customization demands and to meet quality of service requirements. Verification and validation tasks in this context are costly, since they rely on the manual intervention of domain experts at some points of the process. This is mainly due to the difficulty to assess whether the elevators behave as expected in the different test scenarios, the so-called test oracle problem. Metamorphic testing is a thriving testing technique that alleviates the oracle problem by reasoning on the relations among multiple executions of the system under test, the so-called metamorphic relations. In this practical experience paper, we report on the application of metamorphic testing to verify an industrial elevator dispatcher. Together with domain experts from the elevation sector, we defined multiple metamorphic relations that consider domain-specific quality of service measures. Evaluation results with seeded faults show that the approach is effective at detecting faults automatically.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134143986","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

Detecting Implementation Bugs in Graph Convolutional Network based Node Classifiers 基于节点分类器的图卷积网络实现缺陷检测

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00037

Yiwei Wang, Wei Wang, Yujun Cai, Bryan Hooi, B. Ooi

{"title":"Detecting Implementation Bugs in Graph Convolutional Network based Node Classifiers","authors":"Yiwei Wang, Wei Wang, Yujun Cai, Bryan Hooi, B. Ooi","doi":"10.1109/ISSRE5003.2020.00037","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00037","url":null,"abstract":"Graph convolutional networks (GCNs) have achieved state-of-the-art performance on the task of node classification. However, the performance of GCNs is prone to implementation bugs that do not explicitly produce compile-time or run-time errors but degrade their effectiveness heavily. These bugs are hard to detect, since the way in which the node attributes and graph structures contribute to the outputs is complicated, non-transparent, and not traceable by humans. To address this issue, we propose a systematic approach with formal justifications to detect implementation bugs in GCN based node classifiers. Our approach is based on the idea of Metamorphic Testing, which does not check input-output relations for a single input, but for input-output pairs. To speed up our approach, we design a pipeline system, which synchronizes the workload on CPUs and GPUs adaptively and processes them simultaneously. Our empirical study shows that our approach is able to identify over 80% of the synthetic mutants and two real-world bugs in GCN implementations. In addition, our pipeline system can achieve more than $10times$ speedup over the sequential system that leaves the CPU/GPU idle when using the other.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130874074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

Understanding Merge Conflicts and Resolutions in Git Rebases 理解Git rebase中的合并冲突和解决方法

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00016

Tao Ji, Liqian Chen, Xin Yi, Xiaoguang Mao

{"title":"Understanding Merge Conflicts and Resolutions in Git Rebases","authors":"Tao Ji, Liqian Chen, Xin Yi, Xiaoguang Mao","doi":"10.1109/ISSRE5003.2020.00016","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00016","url":null,"abstract":"Software merging is an important activity during software development. Merge conflicts may arise and degrade the software quality. Empirical studies on software merging are helpful to understand developers’ needs and the challenges of detecting and resolving conflicts. Existing studies collect merges by identifying commits that have more than one parent commit. Different from these explicit merges, rebasing branches is used to merge other changes but rewrites the evolutionary history. Hence, existing studies fail to identify implicit merges performed by rebasing branches. Consequently, the results of these studies may fail to provide comprehensive insights on software merging. In our study, we leverage the recently updated APIs of GitHub to study rebase activities in the pull requests. Our study shows that rebasing is widely used in pull requests. And our results indicate that, to resolve textual conflicts, developers adopt similar strategies shown in existing studies on explicit merges. However, in 34.2% of non-conflict rebase scenarios, developers add new changes during the rebase process. And this indicates that there are some new challenges of validating rebases. Our results provide useful insights for improving the state-of-the-art techniques on resolving conflicts and validating rebases.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115894480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11

Evaluation of RESTful frameworks under soft errors 软错误下RESTful框架的评估

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00042

F. Cerveira, R. Oliveira, R. Barbosa, H. Madeira

{"title":"Evaluation of RESTful frameworks under soft errors","authors":"F. Cerveira, R. Oliveira, R. Barbosa, H. Madeira","doi":"10.1109/ISSRE5003.2020.00042","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00042","url":null,"abstract":"RESTful frameworks provide a platform for easy deployment, and management of enterprise-level microservices in a scalable and maintainable manner. Like any computer system and its components, RESTful frameworks are susceptible to soft errors, a subset of transient hardware faults that are caused by cosmic rays and package impurities, which can lead to unexpected behaviour from the services that use the frameworks. Failures in these platforms can cause unavailability and unreliability which can lead to major damages including financial or reputation losses to the service providers, and frustration to users who rely on the service provided. Despite soft errors and their impact being a well-studied problem in some fields, such as aeronautics and safety-critical systems, their effect on service frameworks is still uncharacterized. This paper employs fault injection and fuzzing to evaluate how 5 different frameworks behave when affected by soft errors. The obtained results show that using a framework increases the probability of experiencing a failure by an amount that varies from framework to framework and suggest that most failures pose an issue for service availability, which can be relatively easily handled by standard fault tolerance techniques.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121227078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Reactive Batching Strategy of Apache Kafka for Reliable Stream Processing in Real-time Apache Kafka实时可靠流处理的响应式批处理策略

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00028

Han Wu, Zhihao Shang, Guang Peng, K. Wolter

{"title":"A Reactive Batching Strategy of Apache Kafka for Reliable Stream Processing in Real-time","authors":"Han Wu, Zhihao Shang, Guang Peng, K. Wolter","doi":"10.1109/ISSRE5003.2020.00028","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00028","url":null,"abstract":"Modern stream processing systems need to process large volumes of data in real-time. Various stream processing frameworks have been developed and messaging systems are widely applied to transfer streaming data among different applications. As a distributed messaging system with growing popularity, Apache Kafka processes streaming data in small batches for efficiency. However, the robustness of Kafka’s batching method against variable operating conditions is not known. In this paper we study the impact of the batch size on the performance of Kafka. Both configuration parameters, the spatial and temporal batch size, are considered. We build a Kafka testbed using Docker containers to analyze the distribution of Kafka’s end-to-end latency. The experimental results indicate that evaluating the mean latency only is unreliable in the context of real-time systems. In the experiments where network faults are injected, we find that the batch size affects the message loss rate in the presence of an unstable network connection. However, allocating resources for message processing and delivery that will violate the reliability requirements implemented as latency constraints of a real-time system is inefficient To address these challenges we propose a reactive batching strategy. We evaluate our batching strategy in both good and poor network conditions. The results show that the strategy is powerful enough to meet both latency and throughput constraints even when network conditions are variable.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"80 20","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131770062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Identifying and Prioritizing Chaos Experiments by Using Established Risk Analysis Techniques 利用已建立的风险分析技术识别和优先排序混沌实验

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-10-01 DOI: 10.1109/ISSRE5003.2020.00030

Dominik Kesim, A. Hoorn, Sebastian Frank, M. Häussler

{"title":"Identifying and Prioritizing Chaos Experiments by Using Established Risk Analysis Techniques","authors":"Dominik Kesim, A. Hoorn, Sebastian Frank, M. Häussler","doi":"10.1109/ISSRE5003.2020.00030","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00030","url":null,"abstract":"The prevalence of microservice architectures and container orchestration technologies increases the complexity of assessing such systems’ resilience. Chaos engineering is an emerging approach for resilience assessment by testing hypotheses after intentionally injecting faults into a distributed system and observing customer- and business-affecting metrics. As the number of potential risks within a complex system is high, the identification and prioritization of effective and efficient chaos experiments are non-trivial. In the scope of an industrial case study, this work investigates means to identify and prioritize chaos experiments by using established risk analysis techniques known from engineering safety-critical systems, namely i) Fault Tree Analysis, ii) Failure Mode and Effects Analysis, iii) and Computer Hazard and Operability Study. We conducted semi-structured interviews to elicit architectural information and resilience requirements of the case study system. The extracted knowledge was leveraged during the application of the risk analysis techniques. A subset of the identified and prioritized risks was used to create and execute chaos experiments. The risk analysis resulted in over 100 findings and revealed that the system is rather fragile as it comprises a high amount of single points of failure. The chaos experiments revealed further weaknesses for formerly unknown system behavior.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127621052","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Dependability Evaluation of Middleware Technology for Large-scale Distributed Caching 大规模分布式缓存中间件技术的可靠性评估

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-08-16 DOI: 10.1109/ISSRE5003.2020.00029

Domenico Cotroneo, R. Natella, Stefano Rosiello

引用次数: 1

An Empirical Evaluation of GDPR Compliance Violations in Android mHealth Apps Android移动健康应用中GDPR违规行为的实证评估

2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) Pub Date : 2020-08-13 DOI: 10.1109/ISSRE5003.2020.00032

Ming Fan, Le Yu, Sen Chen, Hao Zhou, Xiapu Luo, Shuyue Li, Yang Liu, Jun Liu, Ting Liu

{"title":"An Empirical Evaluation of GDPR Compliance Violations in Android mHealth Apps","authors":"Ming Fan, Le Yu, Sen Chen, Hao Zhou, Xiapu Luo, Shuyue Li, Yang Liu, Jun Liu, Ting Liu","doi":"10.1109/ISSRE5003.2020.00032","DOIUrl":"https://doi.org/10.1109/ISSRE5003.2020.00032","url":null,"abstract":"The purpose of the General Data Protection Regulation (GDPR) is to provide improved privacy protection. If an app controls personal data from users, it needs to be compliant with GDPR. However, GDPR lists general rules rather than exact step-by-step guidelines about how to develop an app that fulfills the requirements. Therefore, there may exist GDPR compliance violations in existing apps, which would pose severe privacy threats to app users. In this paper, we take mobile health applications (mHealth apps) as a peephole to examine the status quo of GDPR compliance in Android apps. We first propose an automated system, named HPDROID, to bridge the semantic gap between the general rules of GDPR and the app implementations by identifying the data practices declared in the app privacy policy and the data relevant behaviors in the app code. Then, based on HPDROID, we detect three kinds of GDPR compliance violations, including the incompleteness of privacy policy, the inconsistency of data collections, and the insecurity of data transmission. We perform an empirical evaluation of 796 mHealth apps. The results reveal that 189 (23.7%) of them do not provide complete privacy policies. Moreover, 59 apps collect sensitive data through different measures, but 46 (77.9%) of them contain at least one inconsistent collection behavior. Even worse, among the 59 apps, only 8 apps try to ensure the transmission security of collected data. However, all of them contain at least one encryption or SSL misuse. Our work exposes severe privacy issues to raise awareness of privacy protection for app users and developers.","PeriodicalId":268556,"journal":{"name":"2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)","volume":"74 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116704384","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 39