基于HPC集群的油藏模拟作业运行时间预测两步估计策略

IF 1.5 4区 计算机科学 Q3 COMPUTER SCIENCE, SOFTWARE ENGINEERING
Alan L. Nunes, Bernardo Gallo, Bruno Lopes, Felipe A. Portella, José Viterbo, Lúcia M. A. Drummond, Luciano Andrade, Miguel de Lima, Paulo J. B. Estrela, Renzo Q. Malini
{"title":"基于HPC集群的油藏模拟作业运行时间预测两步估计策略","authors":"Alan L. Nunes,&nbsp;Bernardo Gallo,&nbsp;Bruno Lopes,&nbsp;Felipe A. Portella,&nbsp;José Viterbo,&nbsp;Lúcia M. A. Drummond,&nbsp;Luciano Andrade,&nbsp;Miguel de Lima,&nbsp;Paulo J. B. Estrela,&nbsp;Renzo Q. Malini","doi":"10.1002/cpe.70026","DOIUrl":null,"url":null,"abstract":"<div>\n \n <p>Modeling petroleum field behavior provides crucial knowledge for risk quantification regarding extraction prospects. Since their processing requires significant computational and storage capabilities, oil companies run reservoir simulation jobs on high-performance computing clusters managed by job managers, for example, Slurm. In this scenario, efficiently using machine learning algorithms to predict the runtime of incoming jobs can improve the effectiveness of cluster resources, such as enhancing the resource usage rate and reducing the jobs queue time. This work analyses diverse machine learning-based predictors built from a real-world Slurm jobs log from Petrobras, a globally renowned Brazilian energy company. Furthermore, a two-step estimation strategy that predicts the duration time interval of reservoir simulation jobs is proposed and assessed, indicating that such estimated runtimes, when employed by job managers in their scheduling decisions, can positively impact the throughput of a real-world batch system.</p>\n </div>","PeriodicalId":55214,"journal":{"name":"Concurrency and Computation-Practice & Experience","volume":"37 4-5","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2025-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Two-Step Estimation Strategy for Predicting Petroleum Reservoir Simulation Jobs Runtime on an HPC Cluster\",\"authors\":\"Alan L. Nunes,&nbsp;Bernardo Gallo,&nbsp;Bruno Lopes,&nbsp;Felipe A. Portella,&nbsp;José Viterbo,&nbsp;Lúcia M. A. Drummond,&nbsp;Luciano Andrade,&nbsp;Miguel de Lima,&nbsp;Paulo J. B. Estrela,&nbsp;Renzo Q. Malini\",\"doi\":\"10.1002/cpe.70026\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n <p>Modeling petroleum field behavior provides crucial knowledge for risk quantification regarding extraction prospects. Since their processing requires significant computational and storage capabilities, oil companies run reservoir simulation jobs on high-performance computing clusters managed by job managers, for example, Slurm. In this scenario, efficiently using machine learning algorithms to predict the runtime of incoming jobs can improve the effectiveness of cluster resources, such as enhancing the resource usage rate and reducing the jobs queue time. This work analyses diverse machine learning-based predictors built from a real-world Slurm jobs log from Petrobras, a globally renowned Brazilian energy company. Furthermore, a two-step estimation strategy that predicts the duration time interval of reservoir simulation jobs is proposed and assessed, indicating that such estimated runtimes, when employed by job managers in their scheduling decisions, can positively impact the throughput of a real-world batch system.</p>\\n </div>\",\"PeriodicalId\":55214,\"journal\":{\"name\":\"Concurrency and Computation-Practice & Experience\",\"volume\":\"37 4-5\",\"pages\":\"\"},\"PeriodicalIF\":1.5000,\"publicationDate\":\"2025-03-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Concurrency and Computation-Practice & Experience\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/cpe.70026\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, SOFTWARE ENGINEERING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Concurrency and Computation-Practice & Experience","FirstCategoryId":"94","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/cpe.70026","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0

摘要

油田行为建模为开采前景的风险量化提供了至关重要的知识。由于它们的处理需要大量的计算和存储能力,石油公司在作业管理器(例如Slurm)管理的高性能计算集群上运行油藏模拟作业。在这种情况下,有效地使用机器学习算法来预测传入作业的运行时,可以提高集群资源的有效性,例如提高资源使用率和减少作业排队时间。这项工作分析了来自全球知名的巴西能源公司Petrobras的真实世界Slurm工作日志的各种基于机器学习的预测器。此外,提出并评估了一种预测油藏模拟作业持续时间间隔的两步估计策略,表明当作业管理器在调度决策中使用这种估计的运行时间时,可以对实际批处理系统的吞吐量产生积极影响。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Two-Step Estimation Strategy for Predicting Petroleum Reservoir Simulation Jobs Runtime on an HPC Cluster

Modeling petroleum field behavior provides crucial knowledge for risk quantification regarding extraction prospects. Since their processing requires significant computational and storage capabilities, oil companies run reservoir simulation jobs on high-performance computing clusters managed by job managers, for example, Slurm. In this scenario, efficiently using machine learning algorithms to predict the runtime of incoming jobs can improve the effectiveness of cluster resources, such as enhancing the resource usage rate and reducing the jobs queue time. This work analyses diverse machine learning-based predictors built from a real-world Slurm jobs log from Petrobras, a globally renowned Brazilian energy company. Furthermore, a two-step estimation strategy that predicts the duration time interval of reservoir simulation jobs is proposed and assessed, indicating that such estimated runtimes, when employed by job managers in their scheduling decisions, can positively impact the throughput of a real-world batch system.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Concurrency and Computation-Practice & Experience
Concurrency and Computation-Practice & Experience 工程技术-计算机:理论方法
CiteScore
5.00
自引率
10.00%
发文量
664
审稿时长
9.6 months
期刊介绍: Concurrency and Computation: Practice and Experience (CCPE) publishes high-quality, original research papers, and authoritative research review papers, in the overlapping fields of: Parallel and distributed computing; High-performance computing; Computational and data science; Artificial intelligence and machine learning; Big data applications, algorithms, and systems; Network science; Ontologies and semantics; Security and privacy; Cloud/edge/fog computing; Green computing; and Quantum computing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信