{"title":"利用 HRRR 评估基于机器学习的对流灾害概率预报:量化灾害可预测性和对训练选择的敏感性","authors":"R. Sobash, David A. Ahijevych","doi":"10.1175/waf-d-23-0221.1","DOIUrl":null,"url":null,"abstract":"\nThe High Resolution Rapid Refresh (HRRR) model provides hourly-updating forecasts of convective-scale phenomena, which can be used to infer the potential for convective hazards (e.g., tornadoes, hail, and wind gusts), across the United States. We used deterministic 2019–2020 HRRR version 4 (HRRRv4) forecasts to train neural networks (NNs) to generate 4-hourly probabilistic convective hazard forecasts (NNPFs) for HRRRv4 initializations in 2021, using storm reports as ground truth. The NNPFs were compared to the skill of a smoothed updraft helicity (UH) baseline to quantify the benefit of the NNs. NNPF skill varied by initialization time and time of day, but were all superior to the UH forecast. NNPFs valid at hours between 18 UTC – 00 UTC were most skillful in aggregate, significantly exceeding the baseline forecast skill. Overnight NNPFs (i.e., valid 06–12 UTC) were least skillful, indicating a diurnal cycle in hazard predictability that was present across all HRRRv4 initializations. We explored the sensitivity of HRRRv4 NNPF skill to NN training choices. Including an additional year of 2021 HRRRv4 forecasts for training slightly improved skill for 2022 HRRRv4 NNPFs, while reducing the training dataset size by 40% using only forecasts with storm reports was not detrimental to forecast skill. Finally, NNs trained with 2018–2020 HRRRv3 forecasts led to a reduction in NNPF skill when applied to 2021 HRRRv4 forecasts. In addition to documenting practical predictability challenges with convective hazard prediction, these findings reinforce the need for a consistent model configuration for optimal results when training NNs and provide best practices when constructing a training dataset with operational convection-allowing model forecasts.","PeriodicalId":49369,"journal":{"name":"Weather and Forecasting","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Evaluating machine learning-based probabilistic convective hazard forecasts using the HRRR: Quantifying hazard predictability and sensitivity to training choices\",\"authors\":\"R. Sobash, David A. Ahijevych\",\"doi\":\"10.1175/waf-d-23-0221.1\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"\\nThe High Resolution Rapid Refresh (HRRR) model provides hourly-updating forecasts of convective-scale phenomena, which can be used to infer the potential for convective hazards (e.g., tornadoes, hail, and wind gusts), across the United States. We used deterministic 2019–2020 HRRR version 4 (HRRRv4) forecasts to train neural networks (NNs) to generate 4-hourly probabilistic convective hazard forecasts (NNPFs) for HRRRv4 initializations in 2021, using storm reports as ground truth. The NNPFs were compared to the skill of a smoothed updraft helicity (UH) baseline to quantify the benefit of the NNs. NNPF skill varied by initialization time and time of day, but were all superior to the UH forecast. NNPFs valid at hours between 18 UTC – 00 UTC were most skillful in aggregate, significantly exceeding the baseline forecast skill. Overnight NNPFs (i.e., valid 06–12 UTC) were least skillful, indicating a diurnal cycle in hazard predictability that was present across all HRRRv4 initializations. We explored the sensitivity of HRRRv4 NNPF skill to NN training choices. Including an additional year of 2021 HRRRv4 forecasts for training slightly improved skill for 2022 HRRRv4 NNPFs, while reducing the training dataset size by 40% using only forecasts with storm reports was not detrimental to forecast skill. Finally, NNs trained with 2018–2020 HRRRv3 forecasts led to a reduction in NNPF skill when applied to 2021 HRRRv4 forecasts. In addition to documenting practical predictability challenges with convective hazard prediction, these findings reinforce the need for a consistent model configuration for optimal results when training NNs and provide best practices when constructing a training dataset with operational convection-allowing model forecasts.\",\"PeriodicalId\":49369,\"journal\":{\"name\":\"Weather and Forecasting\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.0000,\"publicationDate\":\"2024-07-24\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Weather and Forecasting\",\"FirstCategoryId\":\"89\",\"ListUrlMain\":\"https://doi.org/10.1175/waf-d-23-0221.1\",\"RegionNum\":3,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"METEOROLOGY & ATMOSPHERIC SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Weather and Forecasting","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1175/waf-d-23-0221.1","RegionNum":3,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"METEOROLOGY & ATMOSPHERIC SCIENCES","Score":null,"Total":0}
引用次数: 0
摘要
高分辨率快速刷新(HRRR)模式提供每小时更新的对流尺度现象预报,可用于推断美国各地发生对流灾害(如龙卷风、冰雹和阵风)的可能性。我们使用确定性的 2019-2020 年 HRRR 第 4 版(HRRRv4)预报训练神经网络(NN),以风暴报告为基本事实,为 2021 年 HRRRv4 初始化生成 4 小时概率对流灾害预报(NNPF)。将 NNPF 与平滑上升气流切变(UH)基线的技能进行比较,以量化 NN 的效益。NNPF 的技能因初始化时间和时间而异,但都优于 UH 预报。在 18 UTC - 00 UTC 时段有效的 NNPF 总体技能最高,大大超过了基线预报技能。过夜的 NNPFs(即 06-12 UTC 时有效)预报技能最低,这表明在所有 HRRRv4 初始化中都存在灾害可预测性的昼夜周期。我们探讨了 HRRRv4 NNPF 技能对 NN 训练选择的敏感性。在 2022 年 HRRRv4 NNPF 的训练中,增加一年的 2021 年 HRRRv4 预测略微提高了预测技能,而仅使用有风暴报告的预测将训练数据集的规模减少 40%,对预测技能并无不利影响。最后,使用 2018-2020 年 HRRRv3 预报训练的 NN 在应用于 2021 年 HRRRv4 预报时,NNPF 技能有所下降。除了记录对流灾害预测的实际可预测性挑战外,这些研究结果还强化了在训练 NN 时为获得最佳结果而进行一致的模型配置的必要性,并提供了在使用允许对流的业务模型预测构建训练数据集时的最佳实践。
Evaluating machine learning-based probabilistic convective hazard forecasts using the HRRR: Quantifying hazard predictability and sensitivity to training choices
The High Resolution Rapid Refresh (HRRR) model provides hourly-updating forecasts of convective-scale phenomena, which can be used to infer the potential for convective hazards (e.g., tornadoes, hail, and wind gusts), across the United States. We used deterministic 2019–2020 HRRR version 4 (HRRRv4) forecasts to train neural networks (NNs) to generate 4-hourly probabilistic convective hazard forecasts (NNPFs) for HRRRv4 initializations in 2021, using storm reports as ground truth. The NNPFs were compared to the skill of a smoothed updraft helicity (UH) baseline to quantify the benefit of the NNs. NNPF skill varied by initialization time and time of day, but were all superior to the UH forecast. NNPFs valid at hours between 18 UTC – 00 UTC were most skillful in aggregate, significantly exceeding the baseline forecast skill. Overnight NNPFs (i.e., valid 06–12 UTC) were least skillful, indicating a diurnal cycle in hazard predictability that was present across all HRRRv4 initializations. We explored the sensitivity of HRRRv4 NNPF skill to NN training choices. Including an additional year of 2021 HRRRv4 forecasts for training slightly improved skill for 2022 HRRRv4 NNPFs, while reducing the training dataset size by 40% using only forecasts with storm reports was not detrimental to forecast skill. Finally, NNs trained with 2018–2020 HRRRv3 forecasts led to a reduction in NNPF skill when applied to 2021 HRRRv4 forecasts. In addition to documenting practical predictability challenges with convective hazard prediction, these findings reinforce the need for a consistent model configuration for optimal results when training NNs and provide best practices when constructing a training dataset with operational convection-allowing model forecasts.
期刊介绍:
Weather and Forecasting (WAF) (ISSN: 0882-8156; eISSN: 1520-0434) publishes research that is relevant to operational forecasting. This includes papers on significant weather events, forecasting techniques, forecast verification, model parameterizations, data assimilation, model ensembles, statistical postprocessing techniques, the transfer of research results to the forecasting community, and the societal use and value of forecasts. The scope of WAF includes research relevant to forecast lead times ranging from short-term “nowcasts” through seasonal time scales out to approximately two years.