Jotai: A methodology for the generation of executable C benchmarks

IF 1.8 3区 计算机科学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING
Cecília Conde Kind, Michael Canesche, Fernando Magno Quintão Pereira
{"title":"Jotai: A methodology for the generation of executable C benchmarks","authors":"Cecília Conde Kind,&nbsp;Michael Canesche,&nbsp;Fernando Magno Quintão Pereira","doi":"10.1016/j.cola.2025.101368","DOIUrl":null,"url":null,"abstract":"<div><div>This paper presents a methodology for automatically generating well-defined executable benchmarks in C. The generation process is fully automatic: C files are extracted from open-source repositories and split into compilation units. A type reconstructor infers all the types and declarations required to ensure that functions compile. The generation of inputs is guided by constraints specified via a domain-specific language. This DSL refines the types of functions, for instance, creating relations between integer arguments and the length of buffers. Off-the-shelf tools such as <span>AddressSanitizer</span> and <span>Kcc</span> filter out programs with undefined behavior. To demonstrate applicability, this paper analyzes the dynamic behavior of different collections of benchmarks, some with up to 30 thousand samples, to support several observations: (i) the speedup of optimizations does not follow a normal distribution—a property assumed by statistical tests such as the T-test and the Z-test; (ii) there is strong correlation between number of instructions fetched and running time in x86 and in ARM processors; hence, the former—a non-varying quantity—can be used as a proxy for the latter—a varying quantity—in the autotuning of compilation tasks. The apparatus to generate benchmarks is publicly available. A collection of 18 thousand programs thus produced is also available as a <span>CompilerGym</span>’s dataset.</div></div>","PeriodicalId":48552,"journal":{"name":"Journal of Computer Languages","volume":"85 ","pages":"Article 101368"},"PeriodicalIF":1.8000,"publicationDate":"2025-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computer Languages","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2590118425000541","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0

Abstract

This paper presents a methodology for automatically generating well-defined executable benchmarks in C. The generation process is fully automatic: C files are extracted from open-source repositories and split into compilation units. A type reconstructor infers all the types and declarations required to ensure that functions compile. The generation of inputs is guided by constraints specified via a domain-specific language. This DSL refines the types of functions, for instance, creating relations between integer arguments and the length of buffers. Off-the-shelf tools such as AddressSanitizer and Kcc filter out programs with undefined behavior. To demonstrate applicability, this paper analyzes the dynamic behavior of different collections of benchmarks, some with up to 30 thousand samples, to support several observations: (i) the speedup of optimizations does not follow a normal distribution—a property assumed by statistical tests such as the T-test and the Z-test; (ii) there is strong correlation between number of instructions fetched and running time in x86 and in ARM processors; hence, the former—a non-varying quantity—can be used as a proxy for the latter—a varying quantity—in the autotuning of compilation tasks. The apparatus to generate benchmarks is publicly available. A collection of 18 thousand programs thus produced is also available as a CompilerGym’s dataset.
Jotai:生成可执行C基准的方法
本文提出了一种在C语言中自动生成定义良好的可执行基准的方法。生成过程是全自动的:从开源存储库中提取C文件并将其拆分为编译单元。类型重构器推断出确保函数编译所需的所有类型和声明。输入的生成由特定于领域的语言指定的约束来指导。这个DSL细化了函数的类型,例如,创建了整数参数和缓冲区长度之间的关系。现成的工具,如AddressSanitizer和Kcc,可以过滤掉行为未定义的程序。为了证明其适用性,本文分析了不同基准集合的动态行为,其中一些有多达3万个样本,以支持以下几个观察结果:(i)优化的加速不遵循正态分布-这是统计检验(如t检验和z检验)所假设的属性;(ii)在x86和ARM处理器中,获取的指令数与运行时间之间存在很强的相关性;因此,在编译任务的自动调优中,前者(一个不变的量)可以用作后者(一个可变的量)的代理。生成基准测试的工具是公开的。这样产生的18000个程序的集合也可以作为CompilerGym的数据集使用。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Computer Languages
Journal of Computer Languages Computer Science-Computer Networks and Communications
CiteScore
5.00
自引率
13.60%
发文量
36
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信