{"title":"Spirale at the SBFT 2023 Tool Competiton - Cyber-Physical Systems Track","authors":"Domenico De Vivo, A. R. Fasolino","doi":"10.1109/SBFT59156.2023.00007","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00007","url":null,"abstract":"In this paper, we present Spirale, a search-based testing tool designed to generate scenarios for testing Lane-Keeping Assist Systems (LKAS). Spirale took part in the CPS (Cyber-Physical Systems) testing competition held at SBFT 2023.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128071976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Raihana Ferdous, Chia-kang Hung, Fitsum Meshesha Kifetew, D. Prandi, A. Susi
{"title":"EvoMBT at the SBFT 2023 Tool Competition","authors":"Raihana Ferdous, Chia-kang Hung, Fitsum Meshesha Kifetew, D. Prandi, A. Susi","doi":"10.1109/SBFT59156.2023.00018","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00018","url":null,"abstract":"EvoMBT is a model-based test generator that principally uses search algorithms to generate tests from a given extended finite state machine (EFSM). In the context of Cyber-physical systems (CPS) testing, and in particular self-driving cars, we use a model of road configurations from which EvoMBT generates different roads for testing the car. This report introduces EvoMBT and reports the results it achieved in the Cyber-physical systems testing competition at SBFT 2023. EvoMBT was able to trigger several out-of-bound events and stood fourth in the overall ranking consisting of a total of six tools.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129263323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrea Fioraldi, D. Maier, Dongjia Zhang, Addison Crump
{"title":"AFLrustrust: A LibAFL-based AFL++ prototype","authors":"Andrea Fioraldi, D. Maier, Dongjia Zhang, Addison Crump","doi":"10.1109/SBFT59156.2023.00024","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00024","url":null,"abstract":"In this paper, we present a first attempt at rewriting the widely used fuzzer AFL++ as a frontend of LibAFL, our new framework for fuzzers development. This prototype, AFLrustrust as it is written in the Rust programming language, was evaluated in the SBST’23 Fuzzing Competition with great results even being just a first attempt with missing components that are still under development.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128408135","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RIGAA at the SBFT 2023 Tool Competition - Cyber-Physical Systems Track","authors":"D. Humeniuk, Foutse Khomh, G. Antoniol","doi":"10.1109/SBFT59156.2023.00011","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00011","url":null,"abstract":"Testing and verification of autonomous systems is critically important. In the context of SBFT 2023 CPS testing tool competition, we present our tool RIGAA for generating virtual roads to test an autonomous vehicle lane keeping assist system. RIGAA combines reinforcement learning as well as evolutionary search to generate test scenarios. It has achieved the second highest final score among 5 other submitted tools.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121020909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thijs Klooster, F. Turkmen, G. Broenink, Ruben ten Hove, Marcel Böhme
{"title":"Continuous Fuzzing: A Study of the Effectiveness and Scalability of Fuzzing in CI/CD Pipelines","authors":"Thijs Klooster, F. Turkmen, G. Broenink, Ruben ten Hove, Marcel Böhme","doi":"10.1109/SBFT59156.2023.00015","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00015","url":null,"abstract":"While fuzzing can be very costly, it has proven to be a fundamental technique in uncovering bugs (often security related) in many applications. A recent study on bug reports from OSS-Fuzz observed that recent code changes are responsible for 77% of all reported bugs, stressing the importance of continuous testing. With the increased adoption of CI/CD practices in software development, it is only natural to look for effective ways of incorporating fuzzing into continuous security testing. In this paper, we study the effectiveness of fuzz testing in CI/CD pipelines with a focus on security related bugs and seek optimization opportunities to triage commits that do not require fuzzing. Through experimental analysis, we found that the fuzzing effort can be reduced by 63% in three of the nine libraries we analyzed (55% on average). Additionally, we investigate the correlation between fuzzing campaign duration and the effectiveness of fuzzers in vulnerability discovery: a significantly shorter fuzzing campaign facilitates a faster pipeline for developers, while it can still uncover important bugs. Our findings suggest that continuous fuzzing is indeed beneficial for secure software development processes, and that there are many opportunities to improve its effectiveness.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122104736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Antony Bartlett, Cynthia C. S. Liem, Annibale Panichella
{"title":"On the Strengths of Pure Evolutionary Algorithms in Generating Adversarial Examples","authors":"Antony Bartlett, Cynthia C. S. Liem, Annibale Panichella","doi":"10.1109/SBFT59156.2023.00012","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00012","url":null,"abstract":"Deep learning (DL) models are known to be highly accurate, yet vulnerable to adversarial examples. While earlier research focused on generating adversarial examples using white-box strategies, later research focused on black-box strategies, as models often are not accessible to external attackers. Prior studies showed that black-box approaches based on approximate gradient descent algorithms combined with meta-heuristic search (i.e., the BMI-FGSM algorithm) outperform previously proposed white- and black-box strategies. In this paper, we propose a novel black-box approach purely based on differential evolution (DE), i.e., without using any gradient approximation method. In particular, we propose two variants of a customized DE with customized variation operators: (1) a single-objective (Pixel-SOO) variant generating attacks that fool DL models, and (2) a multi-objective variant (Pixel-MOO) that also minimizes the number of changes in generated attacks. Our preliminary study on five canonical image classification models shows that Pixel-SOO and Pixel-MOO are more effective than the state-of-the-art BMI-FGSM in generating adversarial attacks. Furthermore, Pixel-SOO is faster than Pixel-MOO, while the latter produces subtler attacks than its single-objective variant.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127866515","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"CRAG at the SBFT 2023 Tool Competition - Cyber-Physical Systems Track","authors":"Paolo Arcaini, A. Cetinkaya","doi":"10.1109/SBFT59156.2023.00008","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00008","url":null,"abstract":"CRAG is a combinatorial testing-based generator of road scenarios for autonomous driving testing. The parameters of the combinatorial model are used for determining lengths and curvatures of a road in the Frenet frame representation. CRAG first generates a test suite of roads achieving pairwise coverage, and then incrementally searches for roads achieving combinatorial strength k, but only for the k-tuples that contain (k-1)-tuples having optimal test performance in the previously generated roads.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133250625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"HasteFuzz: Full-Speed Fuzzing","authors":"Zhengjie Du, Yuekang Li","doi":"10.1109/SBFT59156.2023.00022","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00022","url":null,"abstract":"Fuzzing throughput is an important factor affecting the overall fuzzing results. HasteFuzz presents an input filtering mechanism to improve fuzzing throughput. During fuzzing, HasteFuzz filters out inputs whose execution traces are encountered before and only conducts further analysis (bug and new coverage checking) on inputs with new execution traces. In this way, HasteFuzz enables the fuzzer to test and analyze more diverse inputs and improve fuzzing throughput.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"92 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128808945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"R-Fuzz at SBFT’2023","authors":"Ju Chen, Chengyu Song, Hengda Yin","doi":"10.1109/SBFT59156.2023.00027","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00027","url":null,"abstract":"In this report, we present R-Fuzz, our hybrid fuzzer tailored for the participation in the fuzzing competition of SBFT’2023.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134422736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"AFLSmart++: Smarter Greybox Fuzzing","authors":"Van-Thuan Pham","doi":"10.1109/SBFT59156.2023.00023","DOIUrl":"https://doi.org/10.1109/SBFT59156.2023.00023","url":null,"abstract":"Model/grammar-based greybox fuzzing has gained attention from both industry and academia due to its capability of discovering bugs/vulnerabilities in programs taking highly-structured inputs. AFLSmart is a specific example. It is a model-based fuzzer that focuses on chunk-based file formats like PNG, PDF and WAV. Its effectiveness is enabled by carefully-designed high-level mutation operators—that work at data chunk levels—and other heuristics such as its validity-based power schedule and deferred cracking mechanism. In this work, we present an extension of AFLSmart in which we explore some design options to (i) support structure-aware low-level mutation operators—that work at bit-byte-word-dword levels—and (ii) improve AFLSmart’s usability and applicability with the so-called composite input model. The extension is called AFLSmart++ and it was evaluated independently—along with 11 other fuzzers—on the Google FuzzBench in a large-scale competition setup. The results show that AFLSmart++ secures the 3rd place in terms of bug finding but it ranks 11th based on its code coverage achievement.","PeriodicalId":309629,"journal":{"name":"2023 IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128048595","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}