多目标拟合q -迭代：增强权单纯形探索的随机抽样方法

Q3 Engineering

IFAC-PapersOnLine Pub Date : 2025-01-01 DOI:10.1016/j.ifacol.2025.08.160

Emiliano Longo , Davide Spinelli , Matteo Giuliani , Andrea Castelletti

{"title":"多目标拟合q -迭代：增强权单纯形探索的随机抽样方法","authors":"Emiliano Longo , Davide Spinelli , Matteo Giuliani , Andrea Castelletti","doi":"10.1016/j.ifacol.2025.08.160","DOIUrl":null,"url":null,"abstract":"<div><div>This work explores the application of the Multi-Objective Fitted Q-Iteration (MOFQI) algorithm in water systems control, and introduces the random sampling to maximise the exploration of the weights simplex. MOFQI, as an offline, model-free, and multi-objective control algorithm, mitigates the challenges of dimensionality, modeling complexity, and multiple objectives encountered by traditional dynamic programming approaches. We explore the construction of the training dataset and leverage the Extra-Trees regressor for the continuous Q-function approximation. We compared MOFQI to Stochastic Dynamic Programming (SDP) on the Lake Como case study, which is characterized by three conflicting objectives: flood and drought control and water supply.</div></div>","PeriodicalId":37894,"journal":{"name":"IFAC-PapersOnLine","volume":"59 9","pages":"Pages 339-344"},"PeriodicalIF":0.0000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multi-Objective Fitted Q-Iteration: Random sampling for enhanced weights simplex exploration⁎\",\"authors\":\"Emiliano Longo , Davide Spinelli , Matteo Giuliani , Andrea Castelletti\",\"doi\":\"10.1016/j.ifacol.2025.08.160\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>This work explores the application of the Multi-Objective Fitted Q-Iteration (MOFQI) algorithm in water systems control, and introduces the random sampling to maximise the exploration of the weights simplex. MOFQI, as an offline, model-free, and multi-objective control algorithm, mitigates the challenges of dimensionality, modeling complexity, and multiple objectives encountered by traditional dynamic programming approaches. We explore the construction of the training dataset and leverage the Extra-Trees regressor for the continuous Q-function approximation. We compared MOFQI to Stochastic Dynamic Programming (SDP) on the Lake Como case study, which is characterized by three conflicting objectives: flood and drought control and water supply.</div></div>\",\"PeriodicalId\":37894,\"journal\":{\"name\":\"IFAC-PapersOnLine\",\"volume\":\"59 9\",\"pages\":\"Pages 339-344\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IFAC-PapersOnLine\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2405896325007505\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"Engineering\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IFAC-PapersOnLine","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2405896325007505","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Engineering","Score":null,"Total":0}

引用次数: 0

摘要

本文探讨了多目标拟合q -迭代（MOFQI）算法在水系统控制中的应用，并引入了随机抽样来最大限度地探索权单纯形。MOFQI作为一种离线、无模型和多目标控制算法，减轻了传统动态规划方法所面临的维数、建模复杂性和多目标的挑战。我们探索了训练数据集的构建，并利用Extra-Trees回归器进行连续q函数逼近。我们将MOFQI与随机动态规划（SDP）在科莫湖的案例研究中进行了比较，后者的特点是三个相互冲突的目标：水旱控制和供水。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Multi-Objective Fitted Q-Iteration: Random sampling for enhanced weights simplex exploration⁎

This work explores the application of the Multi-Objective Fitted Q-Iteration (MOFQI) algorithm in water systems control, and introduces the random sampling to maximise the exploration of the weights simplex. MOFQI, as an offline, model-free, and multi-objective control algorithm, mitigates the challenges of dimensionality, modeling complexity, and multiple objectives encountered by traditional dynamic programming approaches. We explore the construction of the training dataset and leverage the Extra-Trees regressor for the continuous Q-function approximation. We compared MOFQI to Stochastic Dynamic Programming (SDP) on the Lake Como case study, which is characterized by three conflicting objectives: flood and drought control and water supply.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

IFAC-PapersOnLine Engineering-Control and Systems Engineering

CiteScore

1.70

自引率

0.00%

发文量

1122

期刊介绍： All papers from IFAC meetings are published, in partnership with Elsevier, the IFAC Publisher, in theIFAC-PapersOnLine proceedings series hosted at the ScienceDirect web service. This series includes papers previously published in the IFAC website.The main features of the IFAC-PapersOnLine series are: -Online archive including papers from IFAC Symposia, Congresses, Conferences, and most Workshops. -All papers accepted at the meeting are published in PDF format - searchable and citable. -All papers published on the web site can be cited using the IFAC PapersOnLine ISSN and the individual paper DOI (Digital Object Identifier). The site is Open Access in nature - no charge is made to individuals for reading or downloading. Copyright of all papers belongs to IFAC and must be referenced if derivative journal papers are produced from the conference papers. All papers published in IFAC-PapersOnLine have undergone a peer review selection process according to the IFAC rules.