A linear programming-based hyper local search for tuning hyperparameters

IF 0.9 4区管理学 Q4 OPERATIONS RESEARCH & MANAGEMENT SCIENCE

Operations Research Letters Pub Date : 2025-04-02 DOI:10.1016/j.orl.2025.107287

Ankur Sinha , Satender Gunwal

引用次数: 0

Abstract

We introduce a linear programming-based approach for hyperparameter tuning of machine learning models. The approach finetunes continuous hyperparameters and model parameters through a linear program, enhancing model generalization in the vicinity of an initial model. The proposed method converts hyperparameter optimization into a bilevel program and identifies a descent direction to improve validation loss. The results demonstrate improvements in most cases across regression, machine learning, and deep learning tasks, with test performance enhancements ranging from 0.3% to 28.1%.

查看原文本刊更多论文

基于线性规划的超局部寻优超参数

我们介绍了一种基于线性规划的机器学习模型超参数整定方法。该方法通过线性规划对连续超参数和模型参数进行微调，增强了模型在初始模型附近的泛化能力。该方法将超参数优化转换为双层规划，并确定下降方向，以减少验证损失。结果表明，在回归、机器学习和深度学习任务的大多数情况下，测试性能的提高幅度从0.3%到28.1%不等。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Operations Research Letters 管理科学-运筹学与管理科学

CiteScore

2.10

自引率

9.10%

发文量

111

审稿时长

83 days

期刊介绍： Operations Research Letters is committed to the rapid review and fast publication of short articles on all aspects of operations research and analytics. Apart from a limitation to eight journal pages, quality, originality, relevance and clarity are the only criteria for selecting the papers to be published. ORL covers the broad field of optimization, stochastic models and game theory. Specific areas of interest include networks, routing, location, queueing, scheduling, inventory, reliability, and financial engineering. We wish to explore interfaces with other fields such as life sciences and health care, artificial intelligence and machine learning, energy distribution, and computational social sciences and humanities. Our traditional strength is in methodology, including theory, modelling, algorithms and computational studies. We also welcome novel applications and concise literature reviews.