BugOss: A benchmark of real-world regression bugs for empirical investigation of regression fuzzing techniques

IF 3.7 2区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

Journal of Systems and Software Pub Date : 2024-06-15 DOI:10.1016/j.jss.2024.112119

Jeewoong Kim, Shin Hong

{"title":"BugOss: A benchmark of real-world regression bugs for empirical investigation of regression fuzzing techniques","authors":"Jeewoong Kim, Shin Hong","doi":"10.1016/j.jss.2024.112119","DOIUrl":null,"url":null,"abstract":"<div>This paper presents the design and the constitution of BugOss, a real-world regression bug benchmark for empirical study of regression fuzzing techniques. To reproduce the actual project context where a regression bug was introduced, each bug case of BugOss pinpoints the exact bug-inducing commit and provides a specific test oracle considering the presence of other co-existing bugs. BugOss currently comprises 20 real-world bug cases from 20 open-source C/C++ projects, which had been reported by the OSS-Fuzz projects and confirmed by the project maintainers. The empirical investigation with two regression fuzzing techniques show that, with the bug cases in BugOss, the regression fuzzing techniques perform differently depending on the given project context. In addition, the experiments imply that BugOss encompasses various cases of regression bugs in real-world, thus the bug cases would be useful for empirically investigating regression fuzzing techniques.</div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":null,"pages":null},"PeriodicalIF":3.7000,"publicationDate":"2024-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Systems and Software","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S016412122400164X","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

This paper presents the design and the constitution of BugOss, a real-world regression bug benchmark for empirical study of regression fuzzing techniques. To reproduce the actual project context where a regression bug was introduced, each bug case of BugOss pinpoints the exact bug-inducing commit and provides a specific test oracle considering the presence of other co-existing bugs. BugOss currently comprises 20 real-world bug cases from 20 open-source C/C++ projects, which had been reported by the OSS-Fuzz projects and confirmed by the project maintainers. The empirical investigation with two regression fuzzing techniques show that, with the bug cases in BugOss, the regression fuzzing techniques perform differently depending on the given project context. In addition, the experiments imply that BugOss encompasses various cases of regression bugs in real-world, thus the bug cases would be useful for empirically investigating regression fuzzing techniques.

查看原文本刊更多论文

BugOss：用于回归模糊技术实证研究的真实世界回归错误基准

本文介绍了 BugOss 的设计和构成，这是一个用于回归模糊技术实证研究的真实世界回归错误基准。为了再现引入回归错误的实际项目环境，BugOss 的每个错误案例都精确定位了引发错误的提交，并在考虑到其他同时存在的错误的情况下提供了特定的测试oracle。BugOss 目前包含来自 20 个开源 C/C++ 项目的 20 个真实错误案例，这些案例由 OSS-Fuzz 项目报告并由项目维护者确认。使用两种回归模糊技术进行的实证调查表明，对于 BugOss 中的错误案例，回归模糊技术会根据给定的项目环境发挥不同的作用。此外，实验还表明 BugOss 包含了现实世界中的各种回归错误案例，因此错误案例对回归模糊技术的实证研究非常有用。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Systems and Software 工程技术-计算机：理论方法

CiteScore

8.60

自引率

5.70%

发文量

193

审稿时长

16 weeks

期刊介绍： The Journal of Systems and Software publishes papers covering all aspects of software engineering and related hardware-software-systems issues. All articles should include a validation of the idea presented, e.g. through case studies, experiments, or systematic comparisons with other approaches already in practice. Topics of interest include, but are not limited to: • Methods and tools for, and empirical studies on, software requirements, design, architecture, verification and validation, maintenance and evolution • Agile, model-driven, service-oriented, open source and global software development • Approaches for mobile, multiprocessing, real-time, distributed, cloud-based, dependable and virtualized systems • Human factors and management concerns of software development • Data management and big data issues of software systems • Metrics and evaluation, data mining of software development resources • Business and economic aspects of software development processes The journal welcomes state-of-the-art surveys and reports of practical experience for all of these topics.