通过部分最优传输的同时异常值排除和分布鲁棒学习

IF 3.9 2区 工程技术 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS
Zhongyu Zhang, Biao Huang, Zukui Li
{"title":"通过部分最优传输的同时异常值排除和分布鲁棒学习","authors":"Zhongyu Zhang,&nbsp;Biao Huang,&nbsp;Zukui Li","doi":"10.1016/j.compchemeng.2025.109408","DOIUrl":null,"url":null,"abstract":"<div><div>Distributionally robust optimization (DRO) is a powerful framework that mitigates the impact of distributional uncertainty. It aims to optimize the worst-case performance over all possible distributions within an ambiguity set, defined around a nominal distribution which is often set as the empirical distribution constructed from data. However, the presence of outliers in the data may distort the construction of the ambiguity set, thereby degrading the performance of DRO. In this work, we propose an integrated approach that combines outlier exclusion and robust model training. Applying partial optimal transport, we identify and retain the subset of samples that contribute to lower model loss, effectively filtering out potential outliers that cause large losses. This retained subset is used to construct the nominal distribution for the Wasserstein DRO formulation, which addresses the residual distributional uncertainty. We derive tractable formulations for both regression and classification problems under this framework and demonstrate its effectiveness through numerical experiments and real-world chemical process datasets. The results demonstrate that the proposed method provides a simple, effective, and implementable solution for robust learning under both outlier contamination and distributional shifts.</div></div>","PeriodicalId":286,"journal":{"name":"Computers & Chemical Engineering","volume":"205 ","pages":"Article 109408"},"PeriodicalIF":3.9000,"publicationDate":"2025-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Simultaneous outlier-exclusion and distributionally robust learning through partial optimal transport\",\"authors\":\"Zhongyu Zhang,&nbsp;Biao Huang,&nbsp;Zukui Li\",\"doi\":\"10.1016/j.compchemeng.2025.109408\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Distributionally robust optimization (DRO) is a powerful framework that mitigates the impact of distributional uncertainty. It aims to optimize the worst-case performance over all possible distributions within an ambiguity set, defined around a nominal distribution which is often set as the empirical distribution constructed from data. However, the presence of outliers in the data may distort the construction of the ambiguity set, thereby degrading the performance of DRO. In this work, we propose an integrated approach that combines outlier exclusion and robust model training. Applying partial optimal transport, we identify and retain the subset of samples that contribute to lower model loss, effectively filtering out potential outliers that cause large losses. This retained subset is used to construct the nominal distribution for the Wasserstein DRO formulation, which addresses the residual distributional uncertainty. We derive tractable formulations for both regression and classification problems under this framework and demonstrate its effectiveness through numerical experiments and real-world chemical process datasets. The results demonstrate that the proposed method provides a simple, effective, and implementable solution for robust learning under both outlier contamination and distributional shifts.</div></div>\",\"PeriodicalId\":286,\"journal\":{\"name\":\"Computers & Chemical Engineering\",\"volume\":\"205 \",\"pages\":\"Article 109408\"},\"PeriodicalIF\":3.9000,\"publicationDate\":\"2025-10-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computers & Chemical Engineering\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0098135425004119\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers & Chemical Engineering","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0098135425004119","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

分布式鲁棒优化(DRO)是一个强大的框架,可以减轻分布不确定性的影响。它旨在优化模糊集内所有可能分布的最坏情况性能,模糊集围绕名义分布定义,名义分布通常被设置为由数据构建的经验分布。然而,数据中异常值的存在可能会扭曲模糊集的构建,从而降低DRO的性能。在这项工作中,我们提出了一种结合异常值排除和鲁棒模型训练的综合方法。应用部分最优传输,我们识别并保留有助于降低模型损失的样本子集,有效地过滤掉导致大损失的潜在异常值。这个保留的子集用于构建Wasserstein DRO公式的名义分布,该公式解决了剩余分布的不确定性。我们在此框架下推导了回归和分类问题的易于处理的公式,并通过数值实验和现实世界的化学过程数据集证明了其有效性。结果表明,该方法为异常值污染和分布变化下的鲁棒学习提供了一种简单、有效和可实现的解决方案。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Simultaneous outlier-exclusion and distributionally robust learning through partial optimal transport
Distributionally robust optimization (DRO) is a powerful framework that mitigates the impact of distributional uncertainty. It aims to optimize the worst-case performance over all possible distributions within an ambiguity set, defined around a nominal distribution which is often set as the empirical distribution constructed from data. However, the presence of outliers in the data may distort the construction of the ambiguity set, thereby degrading the performance of DRO. In this work, we propose an integrated approach that combines outlier exclusion and robust model training. Applying partial optimal transport, we identify and retain the subset of samples that contribute to lower model loss, effectively filtering out potential outliers that cause large losses. This retained subset is used to construct the nominal distribution for the Wasserstein DRO formulation, which addresses the residual distributional uncertainty. We derive tractable formulations for both regression and classification problems under this framework and demonstrate its effectiveness through numerical experiments and real-world chemical process datasets. The results demonstrate that the proposed method provides a simple, effective, and implementable solution for robust learning under both outlier contamination and distributional shifts.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Computers & Chemical Engineering
Computers & Chemical Engineering 工程技术-工程:化工
CiteScore
8.70
自引率
14.00%
发文量
374
审稿时长
70 days
期刊介绍: Computers & Chemical Engineering is primarily a journal of record for new developments in the application of computing and systems technology to chemical engineering problems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信