Amirfarhad Nilizadeh, Gary T. Leavens, C. Pasareanu, Yannic Noller
{"title":"JMLKelinci+: Detecting Semantic Bugs and Covering Branches with Valid Inputs using Coverage-Guided Fuzzing and Runtime Assertion Checking","authors":"Amirfarhad Nilizadeh, Gary T. Leavens, C. Pasareanu, Yannic Noller","doi":"10.1145/3607538","DOIUrl":"https://doi.org/10.1145/3607538","url":null,"abstract":"Testing to detect semantic bugs is essential, especially for critical systems. Coverage-guided fuzzing (CGF) and runtime assertion checking (RAC) are two well-known approaches for detecting semantic bugs. CGF aims to generate test inputs with high code coverage. However, while CGF tools can be equipped with sanitizers to detect a fixed set of semantic bugs, they can otherwise only detect bugs that lead to a crash. Thus, the first problem we address is how to help fuzzers detect previously unknown semantic bugs that do not lead to a crash. Moreover, a CGF tool may not necessarily cover all branches with valid inputs, although invalid inputs are useless for detecting semantic bugs. So, the second problem is how to guide a fuzzer to maximize coverage using only valid inputs. On the other hand, RAC monitors the expected behavior of a program dynamically and can only detect a semantic bug when a valid test input shows that the program does not satisfy its specification. Thus, the third problem is how to provide high-quality test inputs for a RAC that can trigger potential bugs. The combination of a CGF tool and RAC solves these problems and can cover branches with valid inputs and detect semantic bugs effectively. Our study uses RAC to guarantee that only valid inputs reach the program under test using the program’s specified preconditions and it also uses RAC to detect semantic bugs using specified postconditions. A prototype tool was developed for this study, named JMLKelinci+. Our results show that combining a CGF tool with RAC will lead to executing the program under test only with valid inputs and that this technique can effectively detect semantic bugs. Also, this idea improves the feedback given to a CGF tool, enabling it to cover all branches faster in programs with non-trivial preconditions.","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"1 1","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41602782","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dara Ly, N. Kosmatov, F. Loulergue, Julien Signoles
{"title":"Sound Runtime Assertion Checking for Memory Properties via Program Transformation","authors":"Dara Ly, N. Kosmatov, F. Loulergue, Julien Signoles","doi":"10.1145/3605951","DOIUrl":"https://doi.org/10.1145/3605951","url":null,"abstract":"Runtime Assertion Checking (RAC) for expressive specification languages is a non-trivial verification task, that becomes even more complex for memory-related properties of imperative languages with dynamic memory allocation. It is important to ensure the soundness of RAC verdicts, in particular when RAC reports the absence of failures for execution traces. This paper presents a formalization of a program transformation technique for RAC of memory properties for a representative language with pointers and memory operations, including dynamic allocation and deallocation. The generated program instrumentation relies on an axiomatized observation memory model, which is essential to record and monitor memory-related properties. We prove the soundness of RAC verdicts with regard to the semantics of this language.","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":" ","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47744857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, Dominik Wojtczak
{"title":"Multi-objective ω-Regular Reinforcement Learning","authors":"Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, Dominik Wojtczak","doi":"https://dl.acm.org/doi/10.1145/3605950","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3605950","url":null,"abstract":"<p>The expanding role of reinforcement learning (RL) in safety-critical system design has promoted ω-automata as a way to express learning requirements—often non-Markovian—with greater ease of expression and interpretation than scalar reward signals. However, real-world sequential decision making situations often involve multiple, potentially conflicting, objectives. Two dominant approaches to express relative preferences over multiple objectives are: (1) <i>weighted preference</i>, where the decision maker provides scalar weights for various objectives, and (2) <i>lexicographic preference</i>, where the decision maker provides an order over the objectives such that any amount of satisfaction of a higher-ordered objective is preferable to any amount of a lower-ordered one. In this article, we study and develop RL algorithms to compute optimal strategies in Markov decision processes against multiple ω-regular objectives under weighted and lexicographic preferences. We provide a translation from multiple ω-regular objectives to a scalar reward signal that is both <i>faithful</i> (maximising reward means maximising probability of achieving the objectives under the corresponding preference) and <i>effective</i> (RL quickly converges to optimal strategies). We have implemented the translations in a formal reinforcement learning tool, <span>Mungojerrie</span>, and we present an experimental evaluation of our technique on benchmark learning problems.</p>","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"23 1","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138517764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Verifying Cooperatively-Scheduled Runtimes using CSP","authors":"Jan Bækgaard Pedersen, Kevin Chalmers","doi":"https://dl.acm.org/doi/10.1145/3605942","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3605942","url":null,"abstract":"<p>In this paper we present the novel verification of synchronous channel communication and channel alternation (choice) by considering the environment within which our primitives are executing. Our work is in exploring development of a multi-threaded scheduler for a cooperatively scheduled process-oriented language, ProcessJ. We use CSP to produce formal specifications for the implementation of the various parts of the language runtime (scheduler, runtime components, and generated code). We use established CSP specifications that model channel communication and choice as well as the formal verification tool FDR to formally prove that the implementations are correct and behave as expected, when executed by our scheduler (the execution environment). Our approach is novel and not seen in similar research, because we consider the behaviour of the systems we examine under the restrictions imposed by an execution environment (e.g., a runtime system, a scheduler, an operating system, etc.) and show that even with such restrictions the channel communication and alternation work. More specifically, we show correctness when a system is executed by the ProcessJ cooperative scheduler. The main contributions of this work are in the models defined and method undertaken to verify cooperatively channel communication and choice.</p>","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"3 1","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138517745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Verifying Cooperatively-Scheduled Runtimes using CSP","authors":"J. Pedersen, K. Chalmers","doi":"10.1145/3605942","DOIUrl":"https://doi.org/10.1145/3605942","url":null,"abstract":"In this paper we present the novel verification of synchronous channel communication and channel alternation (choice) by considering the environment within which our primitives are executing. Our work is in exploring development of a multi-threaded scheduler for a cooperatively scheduled process-oriented language, ProcessJ. We use CSP to produce formal specifications for the implementation of the various parts of the language runtime (scheduler, runtime components, and generated code). We use established CSP specifications that model channel communication and choice as well as the formal verification tool FDR to formally prove that the implementations are correct and behave as expected, when executed by our scheduler (the execution environment). Our approach is novel and not seen in similar research, because we consider the behaviour of the systems we examine under the restrictions imposed by an execution environment (e.g., a runtime system, a scheduler, an operating system, etc.) and show that even with such restrictions the channel communication and alternation work. More specifically, we show correctness when a system is executed by the ProcessJ cooperative scheduler. The main contributions of this work are in the models defined and method undertaken to verify cooperatively channel communication and choice.","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":" ","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49136051","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Explanatory Denotational Semantics for Complex Event Patterns","authors":"B. Zielinski","doi":"10.1145/3608486","DOIUrl":"https://doi.org/10.1145/3608486","url":null,"abstract":"Recent years brought popularity and importance of complex event processing (CEP) and associated query languages. CEP systems can be hard to understand. It is often non-trivial to determine streams of events matched by a query, and sometimes we may not notice important edge cases. Hence, the desirability of formal semantics permitting reasoning about resulting complex events and checking if actual matchings agree with our intentions follows. In the paper we introduce a pattern language ({sl PatLang} ) with some unique syntactic features related to variable binding. We provide two distinct denotational semantics for ({sl PatLang} ) : Minimal semantics, sufficient to describe when patterns match, and tree semantics, which provides detailed information about subpatterns with which the matched events actually match, i.e., information about interpretation of matched events induced by the pattern matching. The tree semantics is unnecessary for verifying correctness of pattern matching execution. However, we show that neither minimal semantics, nor semantics from the prior work suffices to effectively locate errors in patterns with respect to their intended meaning, and that the additional information provided by the tree semantics is crucial for that purpose. We prove that tree semantics can be mapped to minimal semantics. Finally, we provide some practical evaluation.","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":" ","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48491183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Explanatory Denotational Semantics for Complex Event Patterns","authors":"Bartosz Zieliński","doi":"https://dl.acm.org/doi/10.1145/3608486","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3608486","url":null,"abstract":"<p>Recent years brought popularity and importance of complex event processing (CEP) and associated query languages. CEP systems can be hard to understand. It is often non-trivial to determine streams of events matched by a query, and sometimes we may not notice important edge cases. Hence, the desirability of formal semantics permitting reasoning about resulting complex events and checking if actual matchings agree with our intentions follows. In the paper we introduce a pattern language ({sl PatLang} ) with some unique syntactic features related to variable binding. We provide two distinct denotational semantics for ({sl PatLang} ): <i>Minimal semantics</i>, sufficient to describe when patterns match, and <i>tree semantics</i>, which provides detailed information about subpatterns with which the matched events actually match, i.e., information about interpretation of matched events induced by the pattern matching. The tree semantics is unnecessary for verifying correctness of pattern matching execution. However, we show that neither minimal semantics, nor semantics from the prior work suffices to effectively locate errors in patterns with respect to their intended meaning, and that the additional information provided by the tree semantics is crucial for that purpose. We prove that tree semantics can be mapped to minimal semantics. Finally, we provide some practical evaluation.</p>","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"6 1","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138517747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E. M. Hahn, Mateo Perez, S. Schewe, F. Somenzi, Ashutosh Trivedi, D. Wojtczak
{"title":"Multi-objective ω-Regular Reinforcement Learning","authors":"E. M. Hahn, Mateo Perez, S. Schewe, F. Somenzi, Ashutosh Trivedi, D. Wojtczak","doi":"10.1145/3605950","DOIUrl":"https://doi.org/10.1145/3605950","url":null,"abstract":"The expanding role of reinforcement learning (RL) in safety-critical system design has promoted ω-automata as a way to express learning requirements—often non-Markovian—with greater ease of expression and interpretation than scalar reward signals. However, real-world sequential decision making situations often involve multiple, potentially conflicting, objectives. Two dominant approaches to express relative preferences over multiple objectives are: (1) weighted preference, where the decision maker provides scalar weights for various objectives, and (2) lexicographic preference, where the decision maker provides an order over the objectives such that any amount of satisfaction of a higher-ordered objective is preferable to any amount of a lower-ordered one. In this article, we study and develop RL algorithms to compute optimal strategies in Markov decision processes against multiple ω-regular objectives under weighted and lexicographic preferences. We provide a translation from multiple ω-regular objectives to a scalar reward signal that is both faithful (maximising reward means maximising probability of achieving the objectives under the corresponding preference) and effective (RL quickly converges to optimal strategies). We have implemented the translations in a formal reinforcement learning tool, Mungojerrie, and we present an experimental evaluation of our technique on benchmark learning problems.","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"35 1","pages":"1 - 24"},"PeriodicalIF":1.0,"publicationDate":"2023-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43722069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alexandra Bugariu, Arshavir Ter-Gabrielyan, Peter Müller
{"title":"Identifying Overly Restrictive Matching Patterns in SMT-based Program Verifiers (Extended Version)","authors":"Alexandra Bugariu, Arshavir Ter-Gabrielyan, Peter Müller","doi":"https://dl.acm.org/doi/10.1145/3571748","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3571748","url":null,"abstract":"<p>Universal quantifiers occur frequently in proof obligations produced by program verifiers, for instance, to axiomatize uninterpreted functions and to statically express properties of arrays. SMT-based verifiers typically reason about them via E-matching, an SMT algorithm that requires syntactic matching patterns to guide the quantifier instantiations. Devising good matching patterns is challenging. In particular, overly restrictive patterns may lead to spurious verification errors if the quantifiers needed for proof are not instantiated; they may also conceal unsoundness caused by inconsistent axiomatizations. In this article, we present the first technique that identifies and helps the users and the developers of program verifiers remedy the effects of overly restrictive matching patterns. We designed a novel algorithm to synthesize missing triggering terms required to complete unsatisfiability proofs via E-matching. Tool developers can use this information to refine their matching patterns and prevent similar verification errors, or to fix a detected unsoundness.</p>","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"23 1","pages":""},"PeriodicalIF":1.0,"publicationDate":"2023-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138517741","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nicklas S. Johansen, Lasse B. Kær, Andreas L. Madsen, K. Ø. Nielsen, J. Srba, Rasmus G. Tollund
{"title":"Kaki: Efficient Concurrent Update Synthesis for SDN","authors":"Nicklas S. Johansen, Lasse B. Kær, Andreas L. Madsen, K. Ø. Nielsen, J. Srba, Rasmus G. Tollund","doi":"10.1145/3605952","DOIUrl":"https://doi.org/10.1145/3605952","url":null,"abstract":"Modern computer networks based on the software-defined networking (SDN) paradigm are becoming increasingly complex and often require frequent configuration changes in order to react to traffic fluctuations. It is essential that forwarding policies are preserved not only before and after the configuration update but also at any moment during the inherently distributed execution of such an update. We present Kaki, a Petri game based tool for automatic synthesis of switch batches which can be updated in parallel without violating a given (regular) forwarding policy like waypointing or service chaining. Kaki guarantees to find the minimum number of concurrent batches and supports both splittable and nonsplittable flow forwarding. In order to achieve optimal performance, we introduce two novel optimisation techniques based on static analysis: decomposition into independent subproblems and identification of switches that can be collectively updated in the same batch. These techniques considerably improve the performance of our tool Kaki, relying on TAPAAL’s verification engine for Petri games as its backend. Experiments on a large benchmark of real networks from the Internet Topology Zoo database demonstrate that Kaki outperforms the state-of-the-art tools Netstack and FLIP. Kaki computes concurrent update synthesis significantly faster than Netstack and compared to FLIP, it provides shorter (and provably optimal) concurrent update sequences at similar runtimes.","PeriodicalId":50432,"journal":{"name":"Formal Aspects of Computing","volume":"35 1","pages":"1 - 22"},"PeriodicalIF":1.0,"publicationDate":"2023-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43411543","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}