Hierarchical U-net with re-parameterization technique for spatio-temporal weather forecasting

IF 4.3 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Machine Learning Pub Date : 2024-01-12 DOI:10.1007/s10994-023-06445-3

{"title":"Hierarchical U-net with re-parameterization technique for spatio-temporal weather forecasting","authors":"","doi":"10.1007/s10994-023-06445-3","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Due to the considerable computational demands of physics-based numerical weather prediction, especially when modeling fine-grained spatio-temporal atmospheric phenomena, deep learning methods offer an advantageous approach by leveraging specialized computing devices to accelerate training and significantly reduce computational costs. Consequently, the application of deep learning methods has presented a novel solution in the field of weather forecasting. In this context, we introduce a groundbreaking deep learning-based weather prediction architecture known as Hierarchical U-Net (HU-Net) with re-parameterization techniques. The HU-Net comprises two essential components: a feature extraction module and a U-Net module with re-parameterization techniques. The feature extraction module consists of two branches. First, the global pattern extraction employs adaptive Fourier neural operators and self-attention, well-known for capturing long-term dependencies in the data. Second, the local pattern extraction utilizes convolution operations as fundamental building blocks, highly proficient in modeling local correlations. Moreover, a feature fusion block dynamically combines dual-scale information. The U-Net module adopts RepBlock with re-parameterization techniques as the fundamental building block, enabling efficient and rapid inference. In extensive experiments carried out on the large-scale weather benchmark dataset <em>WeatherBench</em> at a resolution of 1.40625<span> <span>\\(^\\circ \\)</span> </span>, the results demonstrate that our proposed HU-Net outperforms other baseline models in both prediction accuracy and inference time.</p>","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"35 1","pages":""},"PeriodicalIF":4.3000,"publicationDate":"2024-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Machine Learning","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s10994-023-06445-3","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Due to the considerable computational demands of physics-based numerical weather prediction, especially when modeling fine-grained spatio-temporal atmospheric phenomena, deep learning methods offer an advantageous approach by leveraging specialized computing devices to accelerate training and significantly reduce computational costs. Consequently, the application of deep learning methods has presented a novel solution in the field of weather forecasting. In this context, we introduce a groundbreaking deep learning-based weather prediction architecture known as Hierarchical U-Net (HU-Net) with re-parameterization techniques. The HU-Net comprises two essential components: a feature extraction module and a U-Net module with re-parameterization techniques. The feature extraction module consists of two branches. First, the global pattern extraction employs adaptive Fourier neural operators and self-attention, well-known for capturing long-term dependencies in the data. Second, the local pattern extraction utilizes convolution operations as fundamental building blocks, highly proficient in modeling local correlations. Moreover, a feature fusion block dynamically combines dual-scale information. The U-Net module adopts RepBlock with re-parameterization techniques as the fundamental building block, enabling efficient and rapid inference. In extensive experiments carried out on the large-scale weather benchmark dataset WeatherBench at a resolution of 1.40625 \(^\circ \) , the results demonstrate that our proposed HU-Net outperforms other baseline models in both prediction accuracy and inference time.

查看原文本刊更多论文

用于时空天气预报的重参数化技术层次 U 网

摘要由于基于物理的数值天气预报对计算能力有相当高的要求，特别是在对细粒度时空大气现象进行建模时，深度学习方法提供了一种利用专业计算设备加速训练和显著降低计算成本的有利方法。因此，深度学习方法的应用为天气预报领域提供了一种新的解决方案。在此背景下，我们介绍了一种基于深度学习的开创性天气预报架构，即采用重参数化技术的分层 U-Net （HU-Net）。HU-Net 包括两个基本组成部分：特征提取模块和采用重参数化技术的 U-Net 模块。特征提取模块由两个分支组成。首先，全局模式提取采用了自适应傅立叶神经算子和自注意，这在捕捉数据的长期依赖性方面是众所周知的。其次，局部模式提取利用卷积运算作为基本的构建模块，能很好地模拟局部相关性。此外，特征融合模块可动态结合双尺度信息。U-Net 模块采用 RepBlock 和重参数化技术作为基本构建模块，实现了高效快速的推理。在分辨率为1.40625（^\circ \）的大规模天气基准数据集WeatherBench上进行的大量实验结果表明，我们提出的HU-Net在预测精度和推理时间上都优于其他基线模型。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Machine Learning 工程技术-计算机：人工智能

CiteScore

11.00

自引率

2.70%

发文量

162

审稿时长

3 months

期刊介绍： Machine Learning serves as a global platform dedicated to computational approaches in learning. The journal reports substantial findings on diverse learning methods applied to various problems, offering support through empirical studies, theoretical analysis, or connections to psychological phenomena. It demonstrates the application of learning methods to solve significant problems and aims to enhance the conduct of machine learning research with a focus on verifiable and replicable evidence in published papers.