A benchmark-based method for evaluating hyperparameter optimization techniques of neural networks for surface water quality prediction

IF 6.1 2区 环境科学与生态学 Q2 ENGINEERING, ENVIRONMENTAL
Xuan Wang, Yan Dong, Jing Yang, Zhipeng Liu, Jinsuo Lu
{"title":"A benchmark-based method for evaluating hyperparameter optimization techniques of neural networks for surface water quality prediction","authors":"Xuan Wang, Yan Dong, Jing Yang, Zhipeng Liu, Jinsuo Lu","doi":"10.1007/s11783-024-1814-5","DOIUrl":null,"url":null,"abstract":"<p>Neural networks (NNs) have been used extensively in surface water prediction tasks due to computing algorithm improvements and data accumulation. An essential step in developing an NN is the hyperparameter selection. In practice, it is common to manually determine hyperparameters in the studies of NNs in water resources tasks. This may result in considerable randomness and require significant computation time; therefore, hyperparameter optimization (HPO) is essential. This study adopted five representatives of the HPO techniques in the surface water quality prediction tasks, including the grid sampling (GS), random search (RS), genetic algorithm (GA), Bayesian optimization (BO) based on the Gaussian process (GP), and the tree Parzen estimator (TPE). For the evaluation of these techniques, this study proposed a method: first, the optimal hyperparameter value sets achieved by GS were regarded as the benchmark; then, the other HPO techniques were evaluated and compared with the benchmark in convergence, optimization orientation, and consistency of the optimized values. The results indicated that the TPE-based BO algorithm was recommended because it yielded stable convergence, reasonable optimization orientation, and the highest consistency rates with the benchmark values. The optimization consistency rates via TPE for the hyperparameters hidden layers, hidden dimension, learning rate, and batch size were 86.7%, 73.3%, 73.3%, and 80.0%, respectively. Unlike the evaluation of HPO techniques directly based on the prediction performance of the optimized NN in a single HPO test, the proposed benchmark-based HPO evaluation approach is feasible and robust.\n</p>","PeriodicalId":12720,"journal":{"name":"Frontiers of Environmental Science & Engineering","volume":null,"pages":null},"PeriodicalIF":6.1000,"publicationDate":"2024-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers of Environmental Science & Engineering","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.1007/s11783-024-1814-5","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ENVIRONMENTAL","Score":null,"Total":0}
引用次数: 0

Abstract

Neural networks (NNs) have been used extensively in surface water prediction tasks due to computing algorithm improvements and data accumulation. An essential step in developing an NN is the hyperparameter selection. In practice, it is common to manually determine hyperparameters in the studies of NNs in water resources tasks. This may result in considerable randomness and require significant computation time; therefore, hyperparameter optimization (HPO) is essential. This study adopted five representatives of the HPO techniques in the surface water quality prediction tasks, including the grid sampling (GS), random search (RS), genetic algorithm (GA), Bayesian optimization (BO) based on the Gaussian process (GP), and the tree Parzen estimator (TPE). For the evaluation of these techniques, this study proposed a method: first, the optimal hyperparameter value sets achieved by GS were regarded as the benchmark; then, the other HPO techniques were evaluated and compared with the benchmark in convergence, optimization orientation, and consistency of the optimized values. The results indicated that the TPE-based BO algorithm was recommended because it yielded stable convergence, reasonable optimization orientation, and the highest consistency rates with the benchmark values. The optimization consistency rates via TPE for the hyperparameters hidden layers, hidden dimension, learning rate, and batch size were 86.7%, 73.3%, 73.3%, and 80.0%, respectively. Unlike the evaluation of HPO techniques directly based on the prediction performance of the optimized NN in a single HPO test, the proposed benchmark-based HPO evaluation approach is feasible and robust.

Abstract Image

基于基准的地表水质量预测神经网络超参数优化技术评估方法
由于计算算法的改进和数据的积累,神经网络(NN)已被广泛应用于地表水预测任务中。开发神经网络的一个重要步骤是选择超参数。实际上,在研究水资源任务中的神经网络时,通常需要手动确定超参数。这可能会导致相当大的随机性,并需要大量的计算时间;因此,超参数优化(HPO)至关重要。本研究在地表水水质预测任务中采用了五种代表性的 HPO 技术,包括网格采样(GS)、随机搜索(RS)、遗传算法(GA)、基于高斯过程(GP)的贝叶斯优化(BO)和树状 Parzen 估计器(TPE)。为了对这些技术进行评估,本研究提出了一种方法:首先,将 GS 实现的最优超参数值集视为基准;然后,对其他 HPO 技术进行评估,并在收敛性、优化方向和优化值一致性方面与基准进行比较。结果表明,基于 TPE 的 BO 算法收敛性稳定,优化方向合理,与基准值的一致性最高,因此被推荐使用。通过 TPE 对超参数隐藏层、隐藏维度、学习率和批量大小的优化一致性率分别为 86.7%、73.3%、73.3% 和 80.0%。与直接根据优化后的 NN 在单次 HPO 测试中的预测性能来评估 HPO 技术不同,所提出的基于基准的 HPO 评估方法是可行且稳健的。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Frontiers of Environmental Science & Engineering
Frontiers of Environmental Science & Engineering ENGINEERING, ENVIRONMENTAL-ENVIRONMENTAL SCIENCES
CiteScore
10.90
自引率
12.50%
发文量
988
审稿时长
6.1 months
期刊介绍: Frontiers of Environmental Science & Engineering (FESE) is an international journal for researchers interested in a wide range of environmental disciplines. The journal''s aim is to advance and disseminate knowledge in all main branches of environmental science & engineering. The journal emphasizes papers in developing fields, as well as papers showing the interaction between environmental disciplines and other disciplines. FESE is a bi-monthly journal. Its peer-reviewed contents consist of a broad blend of reviews, research papers, policy analyses, short communications, and opinions. Nonscheduled “special issue” and "hot topic", including a review article followed by a couple of related research articles, are organized to publish novel contributions and breaking results on all aspects of environmental field.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信