利用深度强化学习实现混沌系统中的数据同化

IF 4.4 2区 地球科学 Q1 METEOROLOGY & ATMOSPHERIC SCIENCES
Mohamad Abed El Rahman Hammoud, Naila Raboudi, Edriss S. Titi, Omar Knio, Ibrahim Hoteit
{"title":"利用深度强化学习实现混沌系统中的数据同化","authors":"Mohamad Abed El Rahman Hammoud,&nbsp;Naila Raboudi,&nbsp;Edriss S. Titi,&nbsp;Omar Knio,&nbsp;Ibrahim Hoteit","doi":"10.1029/2023MS004178","DOIUrl":null,"url":null,"abstract":"<p>Data assimilation (DA) plays a pivotal role in diverse applications, ranging from weather forecasting to trajectory planning for autonomous vehicles. A prime example is the widely used ensemble Kalman filter (EnKF), which relies on the Kalman filter's linear update equation to correct each of the ensemble forecast member's state with incoming observations. Recent advancements have witnessed the emergence of deep learning approaches in this domain, primarily within a supervised learning framework. However, the adaptability of such models to untrained scenarios remains a challenge. In this study, we introduce a new DA strategy that utilizes reinforcement learning (RL) to apply state corrections using full or partial observations of the state variables. Our investigation focuses on demonstrating this approach to the chaotic Lorenz 63 and 96 systems, where the agent's objective is to maximize the geometric series with terms that are proportional to the negative root-mean-squared error (RMSE) between the observations and corresponding forecast states. Consequently, the agent develops a correction strategy, enhancing model forecasts based on available observations. Our strategy employs a stochastic action policy, enabling a Monte Carlo-based DA framework that relies on randomly sampling the policy to generate an ensemble of assimilated realizations. Numerical results demonstrate that the developed RL algorithm performs favorably when compared to the EnKF. Additionally, we illustrate the agent's capability to assimilate non-Gaussian observations, addressing one of the limitations of the EnKF.</p>","PeriodicalId":14881,"journal":{"name":"Journal of Advances in Modeling Earth Systems","volume":"16 8","pages":""},"PeriodicalIF":4.4000,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1029/2023MS004178","citationCount":"0","resultStr":"{\"title\":\"Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning\",\"authors\":\"Mohamad Abed El Rahman Hammoud,&nbsp;Naila Raboudi,&nbsp;Edriss S. Titi,&nbsp;Omar Knio,&nbsp;Ibrahim Hoteit\",\"doi\":\"10.1029/2023MS004178\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Data assimilation (DA) plays a pivotal role in diverse applications, ranging from weather forecasting to trajectory planning for autonomous vehicles. A prime example is the widely used ensemble Kalman filter (EnKF), which relies on the Kalman filter's linear update equation to correct each of the ensemble forecast member's state with incoming observations. Recent advancements have witnessed the emergence of deep learning approaches in this domain, primarily within a supervised learning framework. However, the adaptability of such models to untrained scenarios remains a challenge. In this study, we introduce a new DA strategy that utilizes reinforcement learning (RL) to apply state corrections using full or partial observations of the state variables. Our investigation focuses on demonstrating this approach to the chaotic Lorenz 63 and 96 systems, where the agent's objective is to maximize the geometric series with terms that are proportional to the negative root-mean-squared error (RMSE) between the observations and corresponding forecast states. Consequently, the agent develops a correction strategy, enhancing model forecasts based on available observations. Our strategy employs a stochastic action policy, enabling a Monte Carlo-based DA framework that relies on randomly sampling the policy to generate an ensemble of assimilated realizations. Numerical results demonstrate that the developed RL algorithm performs favorably when compared to the EnKF. Additionally, we illustrate the agent's capability to assimilate non-Gaussian observations, addressing one of the limitations of the EnKF.</p>\",\"PeriodicalId\":14881,\"journal\":{\"name\":\"Journal of Advances in Modeling Earth Systems\",\"volume\":\"16 8\",\"pages\":\"\"},\"PeriodicalIF\":4.4000,\"publicationDate\":\"2024-08-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1029/2023MS004178\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Advances in Modeling Earth Systems\",\"FirstCategoryId\":\"89\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1029/2023MS004178\",\"RegionNum\":2,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"METEOROLOGY & ATMOSPHERIC SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Advances in Modeling Earth Systems","FirstCategoryId":"89","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1029/2023MS004178","RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"METEOROLOGY & ATMOSPHERIC SCIENCES","Score":null,"Total":0}
引用次数: 0

摘要

从天气预报到自动驾驶汽车的轨迹规划,数据同化(DA)在各种应用中发挥着举足轻重的作用。一个典型的例子是广泛使用的集合卡尔曼滤波器(EnKF),它依赖于卡尔曼滤波器的线性更新方程,用接收到的观测数据修正每个集合预测成员的状态。最近的进步见证了深度学习方法在这一领域的出现,主要是在监督学习框架内。然而,这些模型对未经训练的场景的适应性仍然是一个挑战。在本研究中,我们介绍了一种新的数据分析策略,它利用强化学习(RL),利用对状态变量的全部或部分观察结果进行状态修正。我们的研究重点是在混沌洛伦兹 63 和 96 系统中演示这种方法,其中代理的目标是最大化几何序列,其项与观测值和相应预测状态之间的负均值平方根误差(RMSE)成正比。因此,代理制定了一种修正策略,根据现有观测结果加强模型预测。我们的策略采用了随机行动策略,使基于蒙特卡罗的数据分析框架成为可能,该框架依赖于随机抽样策略来生成同化现实的集合。数值结果表明,与 EnKF 相比,所开发的 RL 算法性能更佳。此外,我们还说明了代理吸收非高斯观测数据的能力,从而解决了 EnKF 的局限性之一。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning

Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning

Data assimilation (DA) plays a pivotal role in diverse applications, ranging from weather forecasting to trajectory planning for autonomous vehicles. A prime example is the widely used ensemble Kalman filter (EnKF), which relies on the Kalman filter's linear update equation to correct each of the ensemble forecast member's state with incoming observations. Recent advancements have witnessed the emergence of deep learning approaches in this domain, primarily within a supervised learning framework. However, the adaptability of such models to untrained scenarios remains a challenge. In this study, we introduce a new DA strategy that utilizes reinforcement learning (RL) to apply state corrections using full or partial observations of the state variables. Our investigation focuses on demonstrating this approach to the chaotic Lorenz 63 and 96 systems, where the agent's objective is to maximize the geometric series with terms that are proportional to the negative root-mean-squared error (RMSE) between the observations and corresponding forecast states. Consequently, the agent develops a correction strategy, enhancing model forecasts based on available observations. Our strategy employs a stochastic action policy, enabling a Monte Carlo-based DA framework that relies on randomly sampling the policy to generate an ensemble of assimilated realizations. Numerical results demonstrate that the developed RL algorithm performs favorably when compared to the EnKF. Additionally, we illustrate the agent's capability to assimilate non-Gaussian observations, addressing one of the limitations of the EnKF.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Advances in Modeling Earth Systems
Journal of Advances in Modeling Earth Systems METEOROLOGY & ATMOSPHERIC SCIENCES-
CiteScore
11.40
自引率
11.80%
发文量
241
审稿时长
>12 weeks
期刊介绍: The Journal of Advances in Modeling Earth Systems (JAMES) is committed to advancing the science of Earth systems modeling by offering high-quality scientific research through online availability and open access licensing. JAMES invites authors and readers from the international Earth systems modeling community. Open access. Articles are available free of charge for everyone with Internet access to view and download. Formal peer review. Supplemental material, such as code samples, images, and visualizations, is published at no additional charge. No additional charge for color figures. Modest page charges to cover production costs. Articles published in high-quality full text PDF, HTML, and XML. Internal and external reference linking, DOI registration, and forward linking via CrossRef.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信