{"title":"A surrogate model-based ESM parameter tuning scientific workflow management framework for HPC","authors":"Liang Hu, Xianwei Wu, Xilong Che","doi":"10.1007/s12145-024-01460-x","DOIUrl":null,"url":null,"abstract":"<p>In the present era, scientific computation is gradually becoming a primary research method, with an increasing number of researchers engaging in simulation studies on various high-performance computing platforms. Scientific workflows play a crucial role in organizing these complex research tasks effectively. However, poorly managed scientific workflows can lead to wastage of HPC computational resources and fail to alleviate the operational burden on researchers. The parameter optimization of Earth System Models (ESM) poses specific challenges due to its complexity, exacerbating these issues. To address these challenges, we propose a scientific workflow management framework for surrogate-based ESM parameter optimization. This framework consists of four layers: the resource layer, which gathers current resource information; the service layer, which provides various components to ensure the accurate execution of workflows; the management layer, which monitors the execution status of workflows; and the software environment interaction layer, which serves as the interface for data exchange between users and the framework. We monitored a team engaged in tuning CAM parameters before and after adopting the framework, and the results showed significant improvements in operation numbers, task execution time, and storage resource consumption after deploying the framework. This validates that our proposed scientific workflow management framework effectively addresses the challenges in user operations and resource management during surrogate-based ESM optimization processes, demonstrating the potential of our framework.</p>","PeriodicalId":49318,"journal":{"name":"Earth Science Informatics","volume":null,"pages":null},"PeriodicalIF":2.7000,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Earth Science Informatics","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1007/s12145-024-01460-x","RegionNum":4,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
In the present era, scientific computation is gradually becoming a primary research method, with an increasing number of researchers engaging in simulation studies on various high-performance computing platforms. Scientific workflows play a crucial role in organizing these complex research tasks effectively. However, poorly managed scientific workflows can lead to wastage of HPC computational resources and fail to alleviate the operational burden on researchers. The parameter optimization of Earth System Models (ESM) poses specific challenges due to its complexity, exacerbating these issues. To address these challenges, we propose a scientific workflow management framework for surrogate-based ESM parameter optimization. This framework consists of four layers: the resource layer, which gathers current resource information; the service layer, which provides various components to ensure the accurate execution of workflows; the management layer, which monitors the execution status of workflows; and the software environment interaction layer, which serves as the interface for data exchange between users and the framework. We monitored a team engaged in tuning CAM parameters before and after adopting the framework, and the results showed significant improvements in operation numbers, task execution time, and storage resource consumption after deploying the framework. This validates that our proposed scientific workflow management framework effectively addresses the challenges in user operations and resource management during surrogate-based ESM optimization processes, demonstrating the potential of our framework.
期刊介绍:
The Earth Science Informatics [ESIN] journal aims at rapid publication of high-quality, current, cutting-edge, and provocative scientific work in the area of Earth Science Informatics as it relates to Earth systems science and space science. This includes articles on the application of formal and computational methods, computational Earth science, spatial and temporal analyses, and all aspects of computer applications to the acquisition, storage, processing, interchange, and visualization of data and information about the materials, properties, processes, features, and phenomena that occur at all scales and locations in the Earth system’s five components (atmosphere, hydrosphere, geosphere, biosphere, cryosphere) and in space (see "About this journal" for more detail). The quarterly journal publishes research, methodology, and software articles, as well as editorials, comments, and book and software reviews. Review articles of relevant findings, topics, and methodologies are also considered.