{"title":"动态仿真的微分方程基础模型中人为设计模型和数据预训练模型的桥梁","authors":"Zezheng Song, Jiaxin Yuan, Haizhao Yang","doi":"10.1002/adts.202500062","DOIUrl":null,"url":null,"abstract":"The fast simulation of dynamical systems is a key challenge in many scientific and engineering applications, such as weather forecasting, disease control, and drug discovery. With the recent success of deep learning, there is increasing interest in using neural networks to solve differential equations in a data-driven manner. However, existing methods are either limited to specific types of differential equations or require large amounts of data for training. This restricts their practicality in many real-world applications, where data is often scarce or expensive to obtain. To address this, a novel multi-modal foundation model, named <b>FMint</b> (<b>F</b>oundation <b>M</b>odel based on <b>In</b>i<b>t</b>ialization) is proposed, to bridge the gap between human-designed and data-driven models for the fast simulation of dynamical systems. Built on a decoder-only transformer architecture with in-context learning, FMint utilizes both numerical and textual data to learn a universal error correction scheme for dynamical systems, using prompted sequences of coarse solutions from traditional solvers. The model is pre-trained on a corpus of 400K ordinary differential equations (ODEs), and extensive experiments are performed on challenging ODEs that exhibit chaotic behavior and of high dimensionality. The results demonstrate the effectiveness of the proposed model in terms of both accuracy and efficiency compared to classical numerical solvers, highlighting FMint's potential as a general-purpose solver for dynamical systems. This approach achieves an accuracy improvement of 1 to 2 orders of magnitude over state-of-the-art dynamical system simulators, and delivers a 5X speedup compared to traditional numerical algorithms. The code for FMint is available at https://github.com/margotyjx/FMint.","PeriodicalId":7219,"journal":{"name":"Advanced Theory and Simulations","volume":"96 1","pages":""},"PeriodicalIF":2.9000,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model for Dynamical Simulation\",\"authors\":\"Zezheng Song, Jiaxin Yuan, Haizhao Yang\",\"doi\":\"10.1002/adts.202500062\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The fast simulation of dynamical systems is a key challenge in many scientific and engineering applications, such as weather forecasting, disease control, and drug discovery. With the recent success of deep learning, there is increasing interest in using neural networks to solve differential equations in a data-driven manner. However, existing methods are either limited to specific types of differential equations or require large amounts of data for training. This restricts their practicality in many real-world applications, where data is often scarce or expensive to obtain. To address this, a novel multi-modal foundation model, named <b>FMint</b> (<b>F</b>oundation <b>M</b>odel based on <b>In</b>i<b>t</b>ialization) is proposed, to bridge the gap between human-designed and data-driven models for the fast simulation of dynamical systems. Built on a decoder-only transformer architecture with in-context learning, FMint utilizes both numerical and textual data to learn a universal error correction scheme for dynamical systems, using prompted sequences of coarse solutions from traditional solvers. The model is pre-trained on a corpus of 400K ordinary differential equations (ODEs), and extensive experiments are performed on challenging ODEs that exhibit chaotic behavior and of high dimensionality. The results demonstrate the effectiveness of the proposed model in terms of both accuracy and efficiency compared to classical numerical solvers, highlighting FMint's potential as a general-purpose solver for dynamical systems. This approach achieves an accuracy improvement of 1 to 2 orders of magnitude over state-of-the-art dynamical system simulators, and delivers a 5X speedup compared to traditional numerical algorithms. The code for FMint is available at https://github.com/margotyjx/FMint.\",\"PeriodicalId\":7219,\"journal\":{\"name\":\"Advanced Theory and Simulations\",\"volume\":\"96 1\",\"pages\":\"\"},\"PeriodicalIF\":2.9000,\"publicationDate\":\"2025-04-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Advanced Theory and Simulations\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://doi.org/10.1002/adts.202500062\",\"RegionNum\":4,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MULTIDISCIPLINARY SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advanced Theory and Simulations","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1002/adts.202500062","RegionNum":4,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MULTIDISCIPLINARY SCIENCES","Score":null,"Total":0}
FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model for Dynamical Simulation
The fast simulation of dynamical systems is a key challenge in many scientific and engineering applications, such as weather forecasting, disease control, and drug discovery. With the recent success of deep learning, there is increasing interest in using neural networks to solve differential equations in a data-driven manner. However, existing methods are either limited to specific types of differential equations or require large amounts of data for training. This restricts their practicality in many real-world applications, where data is often scarce or expensive to obtain. To address this, a novel multi-modal foundation model, named FMint (Foundation Model based on Initialization) is proposed, to bridge the gap between human-designed and data-driven models for the fast simulation of dynamical systems. Built on a decoder-only transformer architecture with in-context learning, FMint utilizes both numerical and textual data to learn a universal error correction scheme for dynamical systems, using prompted sequences of coarse solutions from traditional solvers. The model is pre-trained on a corpus of 400K ordinary differential equations (ODEs), and extensive experiments are performed on challenging ODEs that exhibit chaotic behavior and of high dimensionality. The results demonstrate the effectiveness of the proposed model in terms of both accuracy and efficiency compared to classical numerical solvers, highlighting FMint's potential as a general-purpose solver for dynamical systems. This approach achieves an accuracy improvement of 1 to 2 orders of magnitude over state-of-the-art dynamical system simulators, and delivers a 5X speedup compared to traditional numerical algorithms. The code for FMint is available at https://github.com/margotyjx/FMint.
期刊介绍:
Advanced Theory and Simulations is an interdisciplinary, international, English-language journal that publishes high-quality scientific results focusing on the development and application of theoretical methods, modeling and simulation approaches in all natural science and medicine areas, including:
materials, chemistry, condensed matter physics
engineering, energy
life science, biology, medicine
atmospheric/environmental science, climate science
planetary science, astronomy, cosmology
method development, numerical methods, statistics