Xuan Wang, Yan Dong, Jing Yang, Zhipeng Liu, Jinsuo Lu
{"title":"基于基准的地表水质量预测神经网络超参数优化技术评估方法","authors":"Xuan Wang, Yan Dong, Jing Yang, Zhipeng Liu, Jinsuo Lu","doi":"10.1007/s11783-024-1814-5","DOIUrl":null,"url":null,"abstract":"<p>Neural networks (NNs) have been used extensively in surface water prediction tasks due to computing algorithm improvements and data accumulation. An essential step in developing an NN is the hyperparameter selection. In practice, it is common to manually determine hyperparameters in the studies of NNs in water resources tasks. This may result in considerable randomness and require significant computation time; therefore, hyperparameter optimization (HPO) is essential. This study adopted five representatives of the HPO techniques in the surface water quality prediction tasks, including the grid sampling (GS), random search (RS), genetic algorithm (GA), Bayesian optimization (BO) based on the Gaussian process (GP), and the tree Parzen estimator (TPE). For the evaluation of these techniques, this study proposed a method: first, the optimal hyperparameter value sets achieved by GS were regarded as the benchmark; then, the other HPO techniques were evaluated and compared with the benchmark in convergence, optimization orientation, and consistency of the optimized values. The results indicated that the TPE-based BO algorithm was recommended because it yielded stable convergence, reasonable optimization orientation, and the highest consistency rates with the benchmark values. The optimization consistency rates via TPE for the hyperparameters hidden layers, hidden dimension, learning rate, and batch size were 86.7%, 73.3%, 73.3%, and 80.0%, respectively. Unlike the evaluation of HPO techniques directly based on the prediction performance of the optimized NN in a single HPO test, the proposed benchmark-based HPO evaluation approach is feasible and robust.\n</p>","PeriodicalId":12720,"journal":{"name":"Frontiers of Environmental Science & Engineering","volume":null,"pages":null},"PeriodicalIF":6.1000,"publicationDate":"2024-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A benchmark-based method for evaluating hyperparameter optimization techniques of neural networks for surface water quality prediction\",\"authors\":\"Xuan Wang, Yan Dong, Jing Yang, Zhipeng Liu, Jinsuo Lu\",\"doi\":\"10.1007/s11783-024-1814-5\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Neural networks (NNs) have been used extensively in surface water prediction tasks due to computing algorithm improvements and data accumulation. An essential step in developing an NN is the hyperparameter selection. In practice, it is common to manually determine hyperparameters in the studies of NNs in water resources tasks. This may result in considerable randomness and require significant computation time; therefore, hyperparameter optimization (HPO) is essential. This study adopted five representatives of the HPO techniques in the surface water quality prediction tasks, including the grid sampling (GS), random search (RS), genetic algorithm (GA), Bayesian optimization (BO) based on the Gaussian process (GP), and the tree Parzen estimator (TPE). For the evaluation of these techniques, this study proposed a method: first, the optimal hyperparameter value sets achieved by GS were regarded as the benchmark; then, the other HPO techniques were evaluated and compared with the benchmark in convergence, optimization orientation, and consistency of the optimized values. The results indicated that the TPE-based BO algorithm was recommended because it yielded stable convergence, reasonable optimization orientation, and the highest consistency rates with the benchmark values. The optimization consistency rates via TPE for the hyperparameters hidden layers, hidden dimension, learning rate, and batch size were 86.7%, 73.3%, 73.3%, and 80.0%, respectively. Unlike the evaluation of HPO techniques directly based on the prediction performance of the optimized NN in a single HPO test, the proposed benchmark-based HPO evaluation approach is feasible and robust.\\n</p>\",\"PeriodicalId\":12720,\"journal\":{\"name\":\"Frontiers of Environmental Science & Engineering\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":6.1000,\"publicationDate\":\"2024-01-20\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Frontiers of Environmental Science & Engineering\",\"FirstCategoryId\":\"93\",\"ListUrlMain\":\"https://doi.org/10.1007/s11783-024-1814-5\",\"RegionNum\":2,\"RegionCategory\":\"环境科学与生态学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ENVIRONMENTAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers of Environmental Science & Engineering","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.1007/s11783-024-1814-5","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ENVIRONMENTAL","Score":null,"Total":0}
A benchmark-based method for evaluating hyperparameter optimization techniques of neural networks for surface water quality prediction
Neural networks (NNs) have been used extensively in surface water prediction tasks due to computing algorithm improvements and data accumulation. An essential step in developing an NN is the hyperparameter selection. In practice, it is common to manually determine hyperparameters in the studies of NNs in water resources tasks. This may result in considerable randomness and require significant computation time; therefore, hyperparameter optimization (HPO) is essential. This study adopted five representatives of the HPO techniques in the surface water quality prediction tasks, including the grid sampling (GS), random search (RS), genetic algorithm (GA), Bayesian optimization (BO) based on the Gaussian process (GP), and the tree Parzen estimator (TPE). For the evaluation of these techniques, this study proposed a method: first, the optimal hyperparameter value sets achieved by GS were regarded as the benchmark; then, the other HPO techniques were evaluated and compared with the benchmark in convergence, optimization orientation, and consistency of the optimized values. The results indicated that the TPE-based BO algorithm was recommended because it yielded stable convergence, reasonable optimization orientation, and the highest consistency rates with the benchmark values. The optimization consistency rates via TPE for the hyperparameters hidden layers, hidden dimension, learning rate, and batch size were 86.7%, 73.3%, 73.3%, and 80.0%, respectively. Unlike the evaluation of HPO techniques directly based on the prediction performance of the optimized NN in a single HPO test, the proposed benchmark-based HPO evaluation approach is feasible and robust.
期刊介绍:
Frontiers of Environmental Science & Engineering (FESE) is an international journal for researchers interested in a wide range of environmental disciplines. The journal''s aim is to advance and disseminate knowledge in all main branches of environmental science & engineering. The journal emphasizes papers in developing fields, as well as papers showing the interaction between environmental disciplines and other disciplines.
FESE is a bi-monthly journal. Its peer-reviewed contents consist of a broad blend of reviews, research papers, policy analyses, short communications, and opinions. Nonscheduled “special issue” and "hot topic", including a review article followed by a couple of related research articles, are organized to publish novel contributions and breaking results on all aspects of environmental field.