Interpretable Transformer Neural Network Prediction of Diverse Environmental Time Series Using Weather Forecasts

IF 4.6 1区 地球科学 Q2 ENVIRONMENTAL SCIENCES
Enrique Orozco López, David Kaplan, Anna Linhoss
{"title":"Interpretable Transformer Neural Network Prediction of Diverse Environmental Time Series Using Weather Forecasts","authors":"Enrique Orozco López, David Kaplan, Anna Linhoss","doi":"10.1029/2023wr036337","DOIUrl":null,"url":null,"abstract":"Transformer neural networks (TNNs) have caused a paradigm shift in deep learning domains like natural language processing, gathering immense interest due to their versatility in other fields such as time series forecasting (TSF). Most current TSF applications of TNNs use only historic observations to predict future events, ignoring information available in weather forecasts to inform better predictions, and with little attention given to the interpretability of the model's use of explanatory inputs. This work explores the potential for TNNs to perform TSF across multiple environmental variables (streamflow, stage, water temperature, and salinity) in two ecologically important regions: the Peace River watershed (Florida) and the northern Gulf of Mexico (Louisiana). The TNN was tested and its prediction uncertainty quantified for each response variable from one-to fourteen-day-ahead forecasts using past observations and spatially distributed weather forecasts. A sensitivity analysis (SA) was performed on the trained TNNs' attention weights to identify the relative influence of each input variable on each response variable across prediction windows. Overall model performance ranged from good to very good (0.78 < NSE < 0.99 for all variables and forecast horizons). Through the SA, we found that the TNN was able to learn the physical patterns behind the data, adapt the use of input variables to each forecast, and increasingly use weather forecast information as prediction windows increased. The TNN's excellent performance and flexibility, along with the intuitive interpretability highlighting the logic behind the models' forecasting decision-making process, provide evidence for the applicability of this architecture to other TSF variables and locations.","PeriodicalId":23799,"journal":{"name":"Water Resources Research","volume":null,"pages":null},"PeriodicalIF":4.6000,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Water Resources Research","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1029/2023wr036337","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 0

Abstract

Transformer neural networks (TNNs) have caused a paradigm shift in deep learning domains like natural language processing, gathering immense interest due to their versatility in other fields such as time series forecasting (TSF). Most current TSF applications of TNNs use only historic observations to predict future events, ignoring information available in weather forecasts to inform better predictions, and with little attention given to the interpretability of the model's use of explanatory inputs. This work explores the potential for TNNs to perform TSF across multiple environmental variables (streamflow, stage, water temperature, and salinity) in two ecologically important regions: the Peace River watershed (Florida) and the northern Gulf of Mexico (Louisiana). The TNN was tested and its prediction uncertainty quantified for each response variable from one-to fourteen-day-ahead forecasts using past observations and spatially distributed weather forecasts. A sensitivity analysis (SA) was performed on the trained TNNs' attention weights to identify the relative influence of each input variable on each response variable across prediction windows. Overall model performance ranged from good to very good (0.78 < NSE < 0.99 for all variables and forecast horizons). Through the SA, we found that the TNN was able to learn the physical patterns behind the data, adapt the use of input variables to each forecast, and increasingly use weather forecast information as prediction windows increased. The TNN's excellent performance and flexibility, along with the intuitive interpretability highlighting the logic behind the models' forecasting decision-making process, provide evidence for the applicability of this architecture to other TSF variables and locations.
利用天气预报对不同环境时间序列进行可解释变压器神经网络预测
变压器神经网络(TNN)在自然语言处理等深度学习领域引发了一场范式变革,由于其在时间序列预测(TSF)等其他领域的多功能性,引起了人们的极大兴趣。目前,TNN 的大多数 TSF 应用仅使用历史观测数据来预测未来事件,忽略了天气预报中可用于更好预测的信息,而且很少关注模型使用解释性输入的可解释性。这项工作探索了 TNN 在两个重要生态区域(和平河流域(佛罗里达州)和墨西哥湾北部(路易斯安那州))的多个环境变量(河水流量、水位、水温和盐度)中执行 TSF 的潜力。利用过去的观测数据和空间分布式天气预报,对 TNN 进行了测试,并对每个响应变量的预测不确定性进行了量化。对训练有素的 TNN 的注意力权重进行了敏感性分析(SA),以确定每个输入变量在不同预测窗口对每个响应变量的相对影响。模型的整体性能从良好到非常好(所有变量和预测视角的 0.78 < NSE < 0.99)不等。通过 SA,我们发现 TNN 能够学习数据背后的物理模式,根据每次预测调整输入变量的使用,并随着预测窗口的增加越来越多地使用天气预报信息。TNN 的卓越性能和灵活性,以及突出模型预测决策过程背后逻辑的直观可解释性,为该架构在其他 TSF 变量和地点的适用性提供了证据。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Water Resources Research
Water Resources Research 环境科学-湖沼学
CiteScore
8.80
自引率
13.00%
发文量
599
审稿时长
3.5 months
期刊介绍: Water Resources Research (WRR) is an interdisciplinary journal that focuses on hydrology and water resources. It publishes original research in the natural and social sciences of water. It emphasizes the role of water in the Earth system, including physical, chemical, biological, and ecological processes in water resources research and management, including social, policy, and public health implications. It encompasses observational, experimental, theoretical, analytical, numerical, and data-driven approaches that advance the science of water and its management. Submissions are evaluated for their novelty, accuracy, significance, and broader implications of the findings.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信