{"title":"An Empirical Comparison of Two Different Strategies to Automated Fault Detection: Machine Learning Versus Dynamic Analysis","authors":"Rafig Almaghairbe, M. Roper","doi":"10.1109/ISSREW.2019.00099","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00099","url":null,"abstract":"Software testing is an established method to ensure software quality and reliability, but it is an expensive process. In recent years, the automation of test case generation has received significant attention as a way to reduce costs. However, the oracle problem (a mechanism for determine the (in) correctness of an executed test case) is still major problem which has been largely ignored. Recent work has shown that building a test oracle using the principles of anomaly detection techniques (mainly semisupervised/ unsupervised learning models based on dynamic execution data consisting of an amalgamation of input/output pairs and execution traces) is able to demonstrate a reasonable level of success in automatically detect passing and failing execution [1], [2]. In this paper, we present a comparison study between our machine-learning based approaches and an existing techniques from the specification mining domain (the data invariant detector Daikon [3]). The two approaches are evaluated on a range of midsized systems and compared in terms of their fault detection ability. The results show that in most cases semi-supervised learning techniques perform far better as an automated test classifier than Daikon. However, there is one system for which our strategy struggles and Daikon performed far better. Furthermore, unsupervised learning techniques performed on a par when compared with Daikon in several cases.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134293393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Emilia Cioroaica, F. Giandomenico, T. Kuhn, F. Lonetti, E. Marchetti, J. Jahic, Frank Schnicke
{"title":"Towards Runtime Monitoring for Malicious Behaviors Detection in Smart Ecosystems","authors":"Emilia Cioroaica, F. Giandomenico, T. Kuhn, F. Lonetti, E. Marchetti, J. Jahic, Frank Schnicke","doi":"10.1109/ISSREW.2019.00072","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00072","url":null,"abstract":"Smart Ecosystem reflects in the control decisions of entities of different nature, especially of its software components. Particularly, the malicious behavior requires a more accurate attention. This paper discusses the challenges related to the evaluation of software smart agents and proposes a first solution leveraging the monitoring facilities for a) assuring conformity between the software agent and its digital twin in a real-time evaluation and b) validating decisions of the digital twins during runtime in a predictive simulation.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124860811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Antoaneta Kondeva, Vivek Nigam, H. Ruess, Carmen Cârlan
{"title":"On Computer-Aided Techniques for Supporting Safety and Security Co-Engineering","authors":"Antoaneta Kondeva, Vivek Nigam, H. Ruess, Carmen Cârlan","doi":"10.1109/ISSREW.2019.00095","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00095","url":null,"abstract":"With the increasing system interconnectivity, cyberattacks on safety-critical systems can lead to catastrophic events. This calls for a better safety and security integration. Indeed, a safety assessment contains security relevant information, such as, key safety hazards, that shall not be triggered by cyber-attacks. Guidelines, such as, SAE J3061 and ED202A, already recommend to exchange information gathered by safety and security engineers during different phases of development. However, these guidelines do not specify exactly how and which information shall be exchanged. We propose a methodology for enabling computer aided techniques for extracting security relevant information from safety analysis. In particular, we propose techniques for automatically constructing Attack Trees from safety artefacts such as fault trees, hazard analysis and safety patterns. Lastly, we illustrate these techniques on an Industry 4.0 application.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116961799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Isolating Real-Time Safety-Critical Embedded Systems via SGX-Based Lightweight Virtualization","authors":"L. Simone, Giovanni Mazzeo","doi":"10.1109/ISSREW.2019.00089","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00089","url":null,"abstract":"A promising approach for designing critical embedded systems is based on virtualization technologies and multi-core platforms. These enable the deployment of both real-time and general-purpose systems with different criticalities in a single host. Integrating virtualization while also meeting the real-time and isolation requirements is non-trivial, and poses significant challenges especially in terms of certification. In recent years, researchers proposed hardware-assisted solutions to face issues coming from virtualization, and recently the use of Operating System (OS) virtualization as a more lightweight approach. Industries are hampered in leveraging this latter type of virtualization despite the clear benefits it introduces, such as reduced overhead, higher scalability, and effortless certification since there is still lack of approaches to address drawbacks. In this position paper, we propose the usage of Intel's CPU security extension, namely SGX, to enable the adoption of enclaves based on unikernel, a flavor of OS-level virtualization, in the context of real-time systems. We present the advantages of leveraging both the SGX isolation and the unikernel features in order to meet the requirements of safety-critical real-time systems and ease the certification process.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121293685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Mobilio, Matteo Orrù, O. Riganelli, Alessandro Tundo, L. Mariani
{"title":"Anomaly Detection As-a-Service","authors":"M. Mobilio, Matteo Orrù, O. Riganelli, Alessandro Tundo, L. Mariani","doi":"10.1109/ISSREW.2019.00071","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00071","url":null,"abstract":"Cloud systems are complex, large, and dynamic systems whose behavior must be continuously analyzed to timely detect misbehaviors and failures. Although there are solutions to flexibly monitor cloud systems, cost-effectively controlling the anomaly detection logic is still a challenge. In particular, cloud operators may need to quickly change the types of detected anomalies and the scope of anomaly detection, for instance based on observations. This kind of intervention still consists of a largely manual and inefficient ad-hoc effort. In this paper, we present Anomaly Detection as-a-Service (ADaaS), which uses the same as-a-service paradigm often exploited in cloud systems to declarative control the anomaly detection logic. Operators can use ADaaS to specify the set of indicators that must be analyzed and the types of anomalies that must be detected, without having to address any operational aspect. Early results with lightweight detectors show that the presented approach is a promising solution to deliver better control of the anomaly detection logic.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132734890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development","authors":"Jukka Ruohonen, Kalle Rindell","doi":"10.1109/ISSREW.2019.00084","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00084","url":null,"abstract":"Fuzzing has been studied and applied ever since the 1990s. Automated and continuous fuzzing has recently been applied also to open source software projects, including the Linux and BSD kernels. This paper concentrates on the practical aspects of continuous kernel fuzzing in four open source kernels. According to the results, there are over 800 unresolved crashes reported for the four kernels by the syzkaller/syzbot framework. Many of these have been reported relatively long ago. Interestingly, fuzzing-induced bugs have been resolved in the BSD kernels more rapidly. Furthermore, assertions and debug checks, use-after-frees, and general protection faults account for the majority of bug types in the Linux kernel. About 23% of the fixed bugs in the Linux kernel have either went through code review or additional testing. Finally, only code churn provides a weak statistical signal for explaining the associated bug fixing times in the Linux kernel.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"352 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122848134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems","authors":"Hiroshi Kuwajima, F. Ishikawa","doi":"10.1109/ISSREW.2019.00035","DOIUrl":"https://doi.org/10.1109/ISSREW.2019.00035","url":null,"abstract":"More and more software practitioners are tackling towards industrial applications of artificial intelligence (AI) systems, especially those based on machine learning (ML). However, many of existing principles and approaches to traditional software systems do not work effectively for the system behavior obtained by training not by logical design. In addition, unique kinds of requirements are emerging such as fairness and explainability. To provide clear guidance to understand and tackle these difficulties, we present an analysis on what quality concepts we should evaluate for AI systems. We base our discussion on ISO/IEC 25000 series, known as SQuaRE, and identify how it should be adapted for the unique nature of ML and Ethics guidelines for trustworthy AI from European Commission. We thus provide holistic insights for quality of AI systems by incorporating the ML nature and AI ethics to the traditional software quality concepts.","PeriodicalId":166239,"journal":{"name":"2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128641424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}