A Practical, Principled Measure of Fuzzer Appeal: A Preliminary Study

2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS) Pub Date : 2020-12-01 DOI:10.1109/QRS51102.2020.00071

M. Gavrilov, Kyle Dewey, Alex Groce, Davina J. Zamanzadeh, B. Hardekopf

{"title":"A Practical, Principled Measure of Fuzzer Appeal: A Preliminary Study","authors":"M. Gavrilov, Kyle Dewey, Alex Groce, Davina J. Zamanzadeh, B. Hardekopf","doi":"10.1109/QRS51102.2020.00071","DOIUrl":null,"url":null,"abstract":"Fuzzers are important bug-finding tools in both academia and industry. To ensure scientific progress, we need a metric for fuzzer comparison. Bug-based metrics are impractical because (1) the definition of \"bug\" is vague, and (2) mapping bug-revealing inputs to bugs requires extensive domain knowledge.In this paper, we propose an automated method for comparing fuzzers that alleviates these problems. We replace the question \"What bugs can this fuzzer find?\" with \"What changes in program behavior over time can this fuzzer detect?\". Intuitively, fuzzers which find more behavioral changes are likely to find more bugs. However, unlike bugs, behavioral changes are well-defined and readily detectable. Our evaluation, executed on three targets with several fuzzers, shows that our method is consistent with bug-based metrics, but without associated difficulties. While further evaluation is needed to establish superiority, our results show that our method warrants further investigation.","PeriodicalId":301814,"journal":{"name":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2020-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/QRS51102.2020.00071","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 4

Abstract

Fuzzers are important bug-finding tools in both academia and industry. To ensure scientific progress, we need a metric for fuzzer comparison. Bug-based metrics are impractical because (1) the definition of "bug" is vague, and (2) mapping bug-revealing inputs to bugs requires extensive domain knowledge.In this paper, we propose an automated method for comparing fuzzers that alleviates these problems. We replace the question "What bugs can this fuzzer find?" with "What changes in program behavior over time can this fuzzer detect?". Intuitively, fuzzers which find more behavioral changes are likely to find more bugs. However, unlike bugs, behavioral changes are well-defined and readily detectable. Our evaluation, executed on three targets with several fuzzers, shows that our method is consistent with bug-based metrics, but without associated difficulties. While further evaluation is needed to establish superiority, our results show that our method warrants further investigation.

查看原文本刊更多论文

一种实用的、原则性的模糊吸引力测量方法:初步研究

Fuzzers在学术界和工业界都是重要的bug发现工具。为了确保科学进步，我们需要一个模糊比较的度量标准。基于bug的度量是不切实际的，因为(1)“bug”的定义是模糊的，并且(2)将揭示bug的输入映射到bug需要广泛的领域知识。在本文中，我们提出了一种自动比较模糊器的方法，以减轻这些问题。我们将问题“这个模糊器能发现什么bug ?”替换为“随着时间的推移，这个模糊器能检测到程序行为的哪些变化?”直觉上，发现更多行为变化的模糊分析人员可能会发现更多漏洞。然而，与bug不同的是，行为变化是定义良好且易于检测的。我们用几个模糊器在三个目标上执行的评估表明，我们的方法与基于bug的度量是一致的，但没有相关的困难。虽然需要进一步的评价来确定其优越性，但我们的结果表明，我们的方法值得进一步研究。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS)

自引率

0.00%

发文量