{"title":"Interpretable Transformer Neural Network Prediction of Diverse Environmental Time Series Using Weather Forecasts","authors":"Enrique Orozco López, David Kaplan, Anna Linhoss","doi":"10.1029/2023wr036337","DOIUrl":null,"url":null,"abstract":"Transformer neural networks (TNNs) have caused a paradigm shift in deep learning domains like natural language processing, gathering immense interest due to their versatility in other fields such as time series forecasting (TSF). Most current TSF applications of TNNs use only historic observations to predict future events, ignoring information available in weather forecasts to inform better predictions, and with little attention given to the interpretability of the model's use of explanatory inputs. This work explores the potential for TNNs to perform TSF across multiple environmental variables (streamflow, stage, water temperature, and salinity) in two ecologically important regions: the Peace River watershed (Florida) and the northern Gulf of Mexico (Louisiana). The TNN was tested and its prediction uncertainty quantified for each response variable from one-to fourteen-day-ahead forecasts using past observations and spatially distributed weather forecasts. A sensitivity analysis (SA) was performed on the trained TNNs' attention weights to identify the relative influence of each input variable on each response variable across prediction windows. Overall model performance ranged from good to very good (0.78 < NSE < 0.99 for all variables and forecast horizons). Through the SA, we found that the TNN was able to learn the physical patterns behind the data, adapt the use of input variables to each forecast, and increasingly use weather forecast information as prediction windows increased. The TNN's excellent performance and flexibility, along with the intuitive interpretability highlighting the logic behind the models' forecasting decision-making process, provide evidence for the applicability of this architecture to other TSF variables and locations.","PeriodicalId":23799,"journal":{"name":"Water Resources Research","volume":null,"pages":null},"PeriodicalIF":4.6000,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Water Resources Research","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1029/2023wr036337","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
Transformer neural networks (TNNs) have caused a paradigm shift in deep learning domains like natural language processing, gathering immense interest due to their versatility in other fields such as time series forecasting (TSF). Most current TSF applications of TNNs use only historic observations to predict future events, ignoring information available in weather forecasts to inform better predictions, and with little attention given to the interpretability of the model's use of explanatory inputs. This work explores the potential for TNNs to perform TSF across multiple environmental variables (streamflow, stage, water temperature, and salinity) in two ecologically important regions: the Peace River watershed (Florida) and the northern Gulf of Mexico (Louisiana). The TNN was tested and its prediction uncertainty quantified for each response variable from one-to fourteen-day-ahead forecasts using past observations and spatially distributed weather forecasts. A sensitivity analysis (SA) was performed on the trained TNNs' attention weights to identify the relative influence of each input variable on each response variable across prediction windows. Overall model performance ranged from good to very good (0.78 < NSE < 0.99 for all variables and forecast horizons). Through the SA, we found that the TNN was able to learn the physical patterns behind the data, adapt the use of input variables to each forecast, and increasingly use weather forecast information as prediction windows increased. The TNN's excellent performance and flexibility, along with the intuitive interpretability highlighting the logic behind the models' forecasting decision-making process, provide evidence for the applicability of this architecture to other TSF variables and locations.
期刊介绍:
Water Resources Research (WRR) is an interdisciplinary journal that focuses on hydrology and water resources. It publishes original research in the natural and social sciences of water. It emphasizes the role of water in the Earth system, including physical, chemical, biological, and ecological processes in water resources research and management, including social, policy, and public health implications. It encompasses observational, experimental, theoretical, analytical, numerical, and data-driven approaches that advance the science of water and its management. Submissions are evaluated for their novelty, accuracy, significance, and broader implications of the findings.