利用大语言模型进行多模态学习,改进核电站的瞬态识别

IF 3.3 3区 工程技术 Q1 NUCLEAR SCIENCE & TECHNOLOGY
Ben Qi, Jun Sun, Zhe Sui, Xingyu Xiao, Jingang Liang
{"title":"利用大语言模型进行多模态学习,改进核电站的瞬态识别","authors":"Ben Qi,&nbsp;Jun Sun,&nbsp;Zhe Sui,&nbsp;Xingyu Xiao,&nbsp;Jingang Liang","doi":"10.1016/j.pnucene.2024.105421","DOIUrl":null,"url":null,"abstract":"<div><p>Transients are events that cause nuclear power plants (NPPs) to transition from a normal state to an abnormal state, which can lead to severe accidents if not properly handled. Transient identification is crucial for NPPs’ safety and operation. In this paper, we propose a novel multimodal text-time series learning framework(MTTL), the first work to apply a large language model for transient identification. The MTTL consists of self-supervised learning pre-training and zero-shot classification for transient identification. During pre-training, the framework utilizes a large language model(LLM) and a time-series(TS) encoder to fully exploit the rich multimodal information available in NPPs, i.e., to obtain the embeddings of both text data and TS data. The LLM is used to capture the transient knowledge of the NPPs by learning from the text data, and the TS encoder is used to capture the temporal dependencies of the transients by encoding the TS data. Both the LLM and the TS encoder have a linear projection head to map the embeddings into a common space. The similarity between the embeddings of the text and TS data is calculated to minimize the contrastive learning loss and obtain a pre-trained model with rich transient knowledge. During the zero-shot classification, the framework utilizes a pre-trained model to effectively identify real-world NPP transients where the data is different from the pre-trained simulated data. The proposed framework is evaluated on the High-Temperature Reactor-Pebblebed Modules (HTR-PM) plant, and the results demonstrate that the MTTL outperforms several baseline methods, including Transformer, LSTM, and CNN1D. The better zero-shot transient identification capability makes it possible to perform better in real-world NPPs.</p></div>","PeriodicalId":20617,"journal":{"name":"Progress in Nuclear Energy","volume":"177 ","pages":"Article 105421"},"PeriodicalIF":3.3000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multimodal learning using large language models to improve transient identification of nuclear power plants\",\"authors\":\"Ben Qi,&nbsp;Jun Sun,&nbsp;Zhe Sui,&nbsp;Xingyu Xiao,&nbsp;Jingang Liang\",\"doi\":\"10.1016/j.pnucene.2024.105421\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Transients are events that cause nuclear power plants (NPPs) to transition from a normal state to an abnormal state, which can lead to severe accidents if not properly handled. Transient identification is crucial for NPPs’ safety and operation. In this paper, we propose a novel multimodal text-time series learning framework(MTTL), the first work to apply a large language model for transient identification. The MTTL consists of self-supervised learning pre-training and zero-shot classification for transient identification. During pre-training, the framework utilizes a large language model(LLM) and a time-series(TS) encoder to fully exploit the rich multimodal information available in NPPs, i.e., to obtain the embeddings of both text data and TS data. The LLM is used to capture the transient knowledge of the NPPs by learning from the text data, and the TS encoder is used to capture the temporal dependencies of the transients by encoding the TS data. Both the LLM and the TS encoder have a linear projection head to map the embeddings into a common space. The similarity between the embeddings of the text and TS data is calculated to minimize the contrastive learning loss and obtain a pre-trained model with rich transient knowledge. During the zero-shot classification, the framework utilizes a pre-trained model to effectively identify real-world NPP transients where the data is different from the pre-trained simulated data. The proposed framework is evaluated on the High-Temperature Reactor-Pebblebed Modules (HTR-PM) plant, and the results demonstrate that the MTTL outperforms several baseline methods, including Transformer, LSTM, and CNN1D. The better zero-shot transient identification capability makes it possible to perform better in real-world NPPs.</p></div>\",\"PeriodicalId\":20617,\"journal\":{\"name\":\"Progress in Nuclear Energy\",\"volume\":\"177 \",\"pages\":\"Article 105421\"},\"PeriodicalIF\":3.3000,\"publicationDate\":\"2024-09-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Progress in Nuclear Energy\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0149197024003718\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"NUCLEAR SCIENCE & TECHNOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Progress in Nuclear Energy","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0149197024003718","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"NUCLEAR SCIENCE & TECHNOLOGY","Score":null,"Total":0}
引用次数: 0

摘要

瞬态是导致核电站(NPP)从正常状态过渡到异常状态的事件,如果处理不当,可能会导致严重事故。瞬态识别对核电站的安全和运行至关重要。在本文中,我们提出了一个新颖的多模态文本-时间序列学习框架(MTTL),这是首个将大型语言模型应用于瞬态识别的工作。MTTL 包括用于瞬态识别的自监督学习预训练和零点分类。在预训练过程中,该框架利用大型语言模型(LLM)和时间序列(TS)编码器来充分利用 NPP 中丰富的多模态信息,即获得文本数据和 TS 数据的嵌入。LLM 通过从文本数据中学习来捕捉 NPP 的瞬态知识,而 TS 编码器则通过对 TS 数据进行编码来捕捉瞬态的时间依赖性。LLM 和 TS 编码器都有一个线性投影头,将嵌入映射到一个共同的空间。通过计算文本和 TS 数据的嵌入之间的相似性,可以最大限度地减少对比学习损失,并获得一个具有丰富瞬态知识的预训练模型。在零点分类过程中,该框架利用预训练模型有效识别真实世界中与预训练模拟数据不同的国家电力公司瞬态。在高温反应堆-卵石床模块(HTR-PM)电站上对所提出的框架进行了评估,结果表明 MTTL 优于几种基准方法,包括 Transformer、LSTM 和 CNN1D。更好的零点瞬态识别能力使其在实际核电厂中的表现更加出色。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Multimodal learning using large language models to improve transient identification of nuclear power plants

Transients are events that cause nuclear power plants (NPPs) to transition from a normal state to an abnormal state, which can lead to severe accidents if not properly handled. Transient identification is crucial for NPPs’ safety and operation. In this paper, we propose a novel multimodal text-time series learning framework(MTTL), the first work to apply a large language model for transient identification. The MTTL consists of self-supervised learning pre-training and zero-shot classification for transient identification. During pre-training, the framework utilizes a large language model(LLM) and a time-series(TS) encoder to fully exploit the rich multimodal information available in NPPs, i.e., to obtain the embeddings of both text data and TS data. The LLM is used to capture the transient knowledge of the NPPs by learning from the text data, and the TS encoder is used to capture the temporal dependencies of the transients by encoding the TS data. Both the LLM and the TS encoder have a linear projection head to map the embeddings into a common space. The similarity between the embeddings of the text and TS data is calculated to minimize the contrastive learning loss and obtain a pre-trained model with rich transient knowledge. During the zero-shot classification, the framework utilizes a pre-trained model to effectively identify real-world NPP transients where the data is different from the pre-trained simulated data. The proposed framework is evaluated on the High-Temperature Reactor-Pebblebed Modules (HTR-PM) plant, and the results demonstrate that the MTTL outperforms several baseline methods, including Transformer, LSTM, and CNN1D. The better zero-shot transient identification capability makes it possible to perform better in real-world NPPs.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Progress in Nuclear Energy
Progress in Nuclear Energy 工程技术-核科学技术
CiteScore
5.30
自引率
14.80%
发文量
331
审稿时长
3.5 months
期刊介绍: Progress in Nuclear Energy is an international review journal covering all aspects of nuclear science and engineering. In keeping with the maturity of nuclear power, articles on safety, siting and environmental problems are encouraged, as are those associated with economics and fuel management. However, basic physics and engineering will remain an important aspect of the editorial policy. Articles published are either of a review nature or present new material in more depth. They are aimed at researchers and technically-oriented managers working in the nuclear energy field. Please note the following: 1) PNE seeks high quality research papers which are medium to long in length. Short research papers should be submitted to the journal Annals in Nuclear Energy. 2) PNE reserves the right to reject papers which are based solely on routine application of computer codes used to produce reactor designs or explain existing reactor phenomena. Such papers, although worthy, are best left as laboratory reports whereas Progress in Nuclear Energy seeks papers of originality, which are archival in nature, in the fields of mathematical and experimental nuclear technology, including fission, fusion (blanket physics, radiation damage), safety, materials aspects, economics, etc. 3) Review papers, which may occasionally be invited, are particularly sought by the journal in these fields.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信