Is Hyper-Parameter Optimization Different for Software Analytics?

IF 6.5 1区计算机科学 Q1 COMPUTER SCIENCE, SOFTWARE ENGINEERING

IEEE Transactions on Software Engineering Pub Date : 2025-03-11 DOI:10.1109/TSE.2025.3550103

Rahul Yedida;Tim Menzies

{"title":"Is Hyper-Parameter Optimization Different for Software Analytics?","authors":"Rahul Yedida;Tim Menzies","doi":"10.1109/TSE.2025.3550103","DOIUrl":null,"url":null,"abstract":"Yes. SE data can have “smoother” boundaries between classes (compared to traditional AI data sets). To be more precise, the magnitude of the second derivative of the loss function found in SE data is typically much smaller. A new hyper-parameter optimizer, called <monospace>SMOOTHIE</monospace>, can exploit this idiosyncrasy of SE data. We compare <monospace>SMOOTHIE</monospace> and a state-of-the-art AI hyper-parameter optimizer on three tasks: (a) GitHub issue lifetime prediction (b) detecting static code warnings false alarm; (c) defect prediction. For completeness, we also show experiments on some standard AI datasets. <monospace>SMOOTHIE</monospace> runs faster and predicts better on the SE data–but ties on non-SE data with the AI tool. Hence we conclude that SE data can be different to other kinds of data; and those differences mean that we should use different kinds of algorithms for our data. To support open science and other researchers working in this area, all our scripts and datasets are available on-line at <uri>https://github.com/yrahul3910/smoothness-hpo/</uri>.","PeriodicalId":13324,"journal":{"name":"IEEE Transactions on Software Engineering","volume":"51 6","pages":"1629-1644"},"PeriodicalIF":6.5000,"publicationDate":"2025-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Software Engineering","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10919477/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}

引用次数: 0

Abstract

Yes. SE data can have “smoother” boundaries between classes (compared to traditional AI data sets). To be more precise, the magnitude of the second derivative of the loss function found in SE data is typically much smaller. A new hyper-parameter optimizer, called SMOOTHIE, can exploit this idiosyncrasy of SE data. We compare SMOOTHIE and a state-of-the-art AI hyper-parameter optimizer on three tasks: (a) GitHub issue lifetime prediction (b) detecting static code warnings false alarm; (c) defect prediction. For completeness, we also show experiments on some standard AI datasets. SMOOTHIE runs faster and predicts better on the SE data–but ties on non-SE data with the AI tool. Hence we conclude that SE data can be different to other kinds of data; and those differences mean that we should use different kinds of algorithms for our data. To support open science and other researchers working in this area, all our scripts and datasets are available on-line at https://github.com/yrahul3910/smoothness-hpo/.

查看原文本刊更多论文

超参数优化与软件分析不同吗？

是。与传统AI数据集相比，SE数据在类之间具有“更平滑”的边界。更准确地说，在SE数据中发现的损失函数的二阶导数的幅度通常要小得多。一种名为SMOOTHIE的新型超参数优化器可以利用SE数据的这种特性。我们比较了SMOOTHIE和最先进的人工智能超参数优化器在三个任务上：(a) GitHub发布生命周期预测(b)检测静态代码警告假警报；(c)缺陷预测。为了完整起见，我们还展示了在一些标准人工智能数据集上的实验。SMOOTHIE在SE数据上运行得更快，预测得更好，但它将非SE数据与AI工具联系在一起。因此，我们得出结论，SE数据可以不同于其他类型的数据；这些差异意味着我们应该对我们的数据使用不同的算法。为了支持开放科学和在这一领域工作的其他研究人员，我们所有的脚本和数据集都可以在https://github.com/yrahul3910/smoothness-hpo/上在线获得。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Software Engineering 工程技术-工程：电子与电气

CiteScore

9.70

自引率

10.80%

发文量

724

审稿时长

6 months

期刊介绍： IEEE Transactions on Software Engineering seeks contributions comprising well-defined theoretical results and empirical studies with potential impacts on software construction, analysis, or management. The scope of this Transactions extends from fundamental mechanisms to the development of principles and their application in specific environments. Specific topic areas include: a) Development and maintenance methods and models: Techniques and principles for specifying, designing, and implementing software systems, encompassing notations and process models. b) Assessment methods: Software tests, validation, reliability models, test and diagnosis procedures, software redundancy, design for error control, and measurements and evaluation of process and product aspects. c) Software project management: Productivity factors, cost models, schedule and organizational issues, and standards. d) Tools and environments: Specific tools, integrated tool environments, associated architectures, databases, and parallel and distributed processing issues. e) System issues: Hardware-software trade-offs. f) State-of-the-art surveys: Syntheses and comprehensive reviews of the historical development within specific areas of interest.