{"title":"A spatio-temporal fusion deep learning network with application to lightning nowcasting","authors":"Changhai Zhou, Ling Fan, Ferrante Neri","doi":"10.3233/ica-240734","DOIUrl":null,"url":null,"abstract":"Lightning is a rapidly evolving phenomenon, exhibiting both mesoscale and microscale characteristics. Its prediction significantly relies on timely and accurate data observation. With the implementation of new generation weather radar systems and lightning detection networks, radar reflectivity image products, and lightning observation data are becoming increasingly abundant. Research focus has shifted towards lightning nowcasting (prediction of imminent events), utilizing deep learning (DL) methods to extract lightning features from very large data sets. In this paper, we propose a novel spatio-temporal fusion deep learning lightning nowcasting network (STF-LightNet) for lightning nowcasting. The network is based on a 3-dimensional U-Net architecture with encoder-decoder blocks and adopts a structure of multiple branches as well as the main path for the encoder block. To address the challenges of feature extraction and fusion of multi-source data, multiple branches are used to extract different data features independently, and the main path fuses these features. Additionally, a spatial attention (SA) module is added to each branch and the main path to automatically identify lightning areas and enhance their features. The main path fusion is conducted in two steps: the first step fuses features from the branches, and the second fuses features from the previous and current levels of the main path using two different methodsthe weighted summation fusion method and the attention gate fusion method. To overcome the sparsity of lightning observations, we employ an inverse frequency weighted cross-entropy loss function. Finally, STF-LightNet is trained using observations from the previous half hour to predict lightning in the next hour. The outcomes illustrate that the fusion of both the multi-branch and main path structures enhances the network’s ability to effectively integrate features from diverse data sources. Attention mechanisms and fusion modules allow the network to capture more detailed features in the images.","PeriodicalId":50358,"journal":{"name":"Integrated Computer-Aided Engineering","volume":null,"pages":null},"PeriodicalIF":5.8000,"publicationDate":"2024-02-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Integrated Computer-Aided Engineering","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.3233/ica-240734","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Lightning is a rapidly evolving phenomenon, exhibiting both mesoscale and microscale characteristics. Its prediction significantly relies on timely and accurate data observation. With the implementation of new generation weather radar systems and lightning detection networks, radar reflectivity image products, and lightning observation data are becoming increasingly abundant. Research focus has shifted towards lightning nowcasting (prediction of imminent events), utilizing deep learning (DL) methods to extract lightning features from very large data sets. In this paper, we propose a novel spatio-temporal fusion deep learning lightning nowcasting network (STF-LightNet) for lightning nowcasting. The network is based on a 3-dimensional U-Net architecture with encoder-decoder blocks and adopts a structure of multiple branches as well as the main path for the encoder block. To address the challenges of feature extraction and fusion of multi-source data, multiple branches are used to extract different data features independently, and the main path fuses these features. Additionally, a spatial attention (SA) module is added to each branch and the main path to automatically identify lightning areas and enhance their features. The main path fusion is conducted in two steps: the first step fuses features from the branches, and the second fuses features from the previous and current levels of the main path using two different methodsthe weighted summation fusion method and the attention gate fusion method. To overcome the sparsity of lightning observations, we employ an inverse frequency weighted cross-entropy loss function. Finally, STF-LightNet is trained using observations from the previous half hour to predict lightning in the next hour. The outcomes illustrate that the fusion of both the multi-branch and main path structures enhances the network’s ability to effectively integrate features from diverse data sources. Attention mechanisms and fusion modules allow the network to capture more detailed features in the images.
期刊介绍:
Integrated Computer-Aided Engineering (ICAE) was founded in 1993. "Based on the premise that interdisciplinary thinking and synergistic collaboration of disciplines can solve complex problems, open new frontiers, and lead to true innovations and breakthroughs, the cornerstone of industrial competitiveness and advancement of the society" as noted in the inaugural issue of the journal.
The focus of ICAE is the integration of leading edge and emerging computer and information technologies for innovative solution of engineering problems. The journal fosters interdisciplinary research and presents a unique forum for innovative computer-aided engineering. It also publishes novel industrial applications of CAE, thus helping to bring new computational paradigms from research labs and classrooms to reality. Areas covered by the journal include (but are not limited to) artificial intelligence, advanced signal processing, biologically inspired computing, cognitive modeling, concurrent engineering, database management, distributed computing, evolutionary computing, fuzzy logic, genetic algorithms, geometric modeling, intelligent and adaptive systems, internet-based technologies, knowledge discovery and engineering, machine learning, mechatronics, mobile computing, multimedia technologies, networking, neural network computing, object-oriented systems, optimization and search, parallel processing, robotics virtual reality, and visualization techniques.