Penalty hyperparameter optimization with diversity measure for nonnegative low-rank approximation

IF 2.2 2区 数学 Q1 MATHEMATICS, APPLIED
Nicoletta Del Buono , Flavia Esposito , Laura Selicato , Rafał Zdunek
{"title":"Penalty hyperparameter optimization with diversity measure for nonnegative low-rank approximation","authors":"Nicoletta Del Buono ,&nbsp;Flavia Esposito ,&nbsp;Laura Selicato ,&nbsp;Rafał Zdunek","doi":"10.1016/j.apnum.2024.10.002","DOIUrl":null,"url":null,"abstract":"<div><div>Learning tasks are often based on penalized optimization problems in which a sparse solution is desired. This can lead to more interpretative results by identifying a smaller subset of important features or components and reducing the dimensionality of the data representation, as well. In this study, we propose a new method to solve a constrained Frobenius norm-based nonnegative low-rank approximation, and the tuning of the associated penalty hyperparameter, simultaneously. The penalty term added is a particular diversity measure that is more effective for sparseness purposes than other classical norm-based penalties (i.e., <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mn>1</mn></mrow></msub></math></span> or <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></math></span> norms). As it is well known, setting the hyperparameters of an algorithm is not an easy task. Our work drew on developing an optimization method and the corresponding algorithm that simultaneously solves the sparsity-constrained nonnegative approximation problem and optimizes its associated penalty hyperparameters. We test the proposed method by numerical experiments and show its promising results on several synthetic and real datasets.</div></div>","PeriodicalId":8199,"journal":{"name":"Applied Numerical Mathematics","volume":"208 ","pages":"Pages 189-204"},"PeriodicalIF":2.2000,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Numerical Mathematics","FirstCategoryId":"100","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0168927424002708","RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0

Abstract

Learning tasks are often based on penalized optimization problems in which a sparse solution is desired. This can lead to more interpretative results by identifying a smaller subset of important features or components and reducing the dimensionality of the data representation, as well. In this study, we propose a new method to solve a constrained Frobenius norm-based nonnegative low-rank approximation, and the tuning of the associated penalty hyperparameter, simultaneously. The penalty term added is a particular diversity measure that is more effective for sparseness purposes than other classical norm-based penalties (i.e., 1 or 2,1 norms). As it is well known, setting the hyperparameters of an algorithm is not an easy task. Our work drew on developing an optimization method and the corresponding algorithm that simultaneously solves the sparsity-constrained nonnegative approximation problem and optimizes its associated penalty hyperparameters. We test the proposed method by numerical experiments and show its promising results on several synthetic and real datasets.
非负低秩逼近的带有分集度量的惩罚超参数优化
学习任务通常基于需要稀疏解的惩罚优化问题。这可以通过识别更小的重要特征或组件子集以及降低数据表示的维数来获得更具解释性的结果。在这项研究中,我们提出了一种新的方法来解决基于约束Frobenius范数的非负低秩近似,并同时调整相关的惩罚超参数。添加的惩罚项是一种特殊的多样性度量,对于稀疏性目的,它比其他经典的基于规范的惩罚(即,l1或l1规范)更有效。众所周知,设置算法的超参数并不是一件容易的事。我们的工作是开发一种优化方法和相应的算法,同时解决稀疏约束的非负近似问题并优化其相关的惩罚超参数。通过数值实验对该方法进行了验证,并在几个合成数据集和实际数据集上证明了该方法的良好效果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Applied Numerical Mathematics
Applied Numerical Mathematics 数学-应用数学
CiteScore
5.60
自引率
7.10%
发文量
225
审稿时长
7.2 months
期刊介绍: The purpose of the journal is to provide a forum for the publication of high quality research and tutorial papers in computational mathematics. In addition to the traditional issues and problems in numerical analysis, the journal also publishes papers describing relevant applications in such fields as physics, fluid dynamics, engineering and other branches of applied science with a computational mathematics component. The journal strives to be flexible in the type of papers it publishes and their format. Equally desirable are: (i) Full papers, which should be complete and relatively self-contained original contributions with an introduction that can be understood by the broad computational mathematics community. Both rigorous and heuristic styles are acceptable. Of particular interest are papers about new areas of research, in which other than strictly mathematical arguments may be important in establishing a basis for further developments. (ii) Tutorial review papers, covering some of the important issues in Numerical Mathematics, Scientific Computing and their Applications. The journal will occasionally publish contributions which are larger than the usual format for regular papers. (iii) Short notes, which present specific new results and techniques in a brief communication.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信