A spatio-temporal fusion deep learning network with application to lightning nowcasting

IF 5.8 2区 计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Changhai Zhou, Ling Fan, Ferrante Neri
{"title":"A spatio-temporal fusion deep learning network with application to lightning nowcasting","authors":"Changhai Zhou, Ling Fan, Ferrante Neri","doi":"10.3233/ica-240734","DOIUrl":null,"url":null,"abstract":"Lightning is a rapidly evolving phenomenon, exhibiting both mesoscale and microscale characteristics. Its prediction significantly relies on timely and accurate data observation. With the implementation of new generation weather radar systems and lightning detection networks, radar reflectivity image products, and lightning observation data are becoming increasingly abundant. Research focus has shifted towards lightning nowcasting (prediction of imminent events), utilizing deep learning (DL) methods to extract lightning features from very large data sets. In this paper, we propose a novel spatio-temporal fusion deep learning lightning nowcasting network (STF-LightNet) for lightning nowcasting. The network is based on a 3-dimensional U-Net architecture with encoder-decoder blocks and adopts a structure of multiple branches as well as the main path for the encoder block. To address the challenges of feature extraction and fusion of multi-source data, multiple branches are used to extract different data features independently, and the main path fuses these features. Additionally, a spatial attention (SA) module is added to each branch and the main path to automatically identify lightning areas and enhance their features. The main path fusion is conducted in two steps: the first step fuses features from the branches, and the second fuses features from the previous and current levels of the main path using two different methodsthe weighted summation fusion method and the attention gate fusion method. To overcome the sparsity of lightning observations, we employ an inverse frequency weighted cross-entropy loss function. Finally, STF-LightNet is trained using observations from the previous half hour to predict lightning in the next hour. The outcomes illustrate that the fusion of both the multi-branch and main path structures enhances the network’s ability to effectively integrate features from diverse data sources. Attention mechanisms and fusion modules allow the network to capture more detailed features in the images.","PeriodicalId":50358,"journal":{"name":"Integrated Computer-Aided Engineering","volume":"68 1","pages":""},"PeriodicalIF":5.8000,"publicationDate":"2024-02-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Integrated Computer-Aided Engineering","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.3233/ica-240734","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

Lightning is a rapidly evolving phenomenon, exhibiting both mesoscale and microscale characteristics. Its prediction significantly relies on timely and accurate data observation. With the implementation of new generation weather radar systems and lightning detection networks, radar reflectivity image products, and lightning observation data are becoming increasingly abundant. Research focus has shifted towards lightning nowcasting (prediction of imminent events), utilizing deep learning (DL) methods to extract lightning features from very large data sets. In this paper, we propose a novel spatio-temporal fusion deep learning lightning nowcasting network (STF-LightNet) for lightning nowcasting. The network is based on a 3-dimensional U-Net architecture with encoder-decoder blocks and adopts a structure of multiple branches as well as the main path for the encoder block. To address the challenges of feature extraction and fusion of multi-source data, multiple branches are used to extract different data features independently, and the main path fuses these features. Additionally, a spatial attention (SA) module is added to each branch and the main path to automatically identify lightning areas and enhance their features. The main path fusion is conducted in two steps: the first step fuses features from the branches, and the second fuses features from the previous and current levels of the main path using two different methodsthe weighted summation fusion method and the attention gate fusion method. To overcome the sparsity of lightning observations, we employ an inverse frequency weighted cross-entropy loss function. Finally, STF-LightNet is trained using observations from the previous half hour to predict lightning in the next hour. The outcomes illustrate that the fusion of both the multi-branch and main path structures enhances the network’s ability to effectively integrate features from diverse data sources. Attention mechanisms and fusion modules allow the network to capture more detailed features in the images.
应用于闪电预报的时空融合深度学习网络
闪电是一种快速发展的现象,同时具有中尺度和微尺度的特征。对它的预测在很大程度上依赖于及时准确的数据观测。随着新一代天气雷达系统和闪电探测网络的应用,雷达反射率图像产品和闪电观测数据日益丰富。研究重点已转向闪电预报(预测即将发生的事件),利用深度学习(DL)方法从超大数据集中提取闪电特征。在本文中,我们提出了一种用于闪电预报的新型时空融合深度学习闪电预报网络(STF-LightNet)。该网络基于具有编码器-解码器模块的三维 U-Net 架构,采用多分支结构以及编码器模块的主路径。为了应对多源数据特征提取和融合的挑战,多个分支用于独立提取不同的数据特征,而主路径则用于融合这些特征。此外,每个分支和主路径都添加了空间注意力(SA)模块,以自动识别闪电区域并增强其特征。主路径融合分两步进行:第一步融合分支的特征,第二步使用两种不同的方法(加权求和融合法和注意力门融合法)融合主路径前一级和当前一级的特征。为了克服闪电观测数据的稀疏性,我们采用了反频率加权交叉熵损失函数。最后,利用前半小时的观测数据对 STF-LightNet 进行训练,以预测下一小时的闪电。结果表明,多分支结构和主路径结构的融合增强了网络有效整合不同数据源特征的能力。注意机制和融合模块使网络能够捕捉到图像中更多的细节特征。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Integrated Computer-Aided Engineering
Integrated Computer-Aided Engineering 工程技术-工程:综合
CiteScore
9.90
自引率
21.50%
发文量
21
审稿时长
>12 weeks
期刊介绍: Integrated Computer-Aided Engineering (ICAE) was founded in 1993. "Based on the premise that interdisciplinary thinking and synergistic collaboration of disciplines can solve complex problems, open new frontiers, and lead to true innovations and breakthroughs, the cornerstone of industrial competitiveness and advancement of the society" as noted in the inaugural issue of the journal. The focus of ICAE is the integration of leading edge and emerging computer and information technologies for innovative solution of engineering problems. The journal fosters interdisciplinary research and presents a unique forum for innovative computer-aided engineering. It also publishes novel industrial applications of CAE, thus helping to bring new computational paradigms from research labs and classrooms to reality. Areas covered by the journal include (but are not limited to) artificial intelligence, advanced signal processing, biologically inspired computing, cognitive modeling, concurrent engineering, database management, distributed computing, evolutionary computing, fuzzy logic, genetic algorithms, geometric modeling, intelligent and adaptive systems, internet-based technologies, knowledge discovery and engineering, machine learning, mechatronics, mobile computing, multimedia technologies, networking, neural network computing, object-oriented systems, optimization and search, parallel processing, robotics virtual reality, and visualization techniques.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信