The Frequency-Domain Corrected Attention Operator for solving PDEs

Abstract

Two classical Neural Operators are widely used for solving PDEs. One approach utilizes spectral transformations for learning in the spectral domain, while the other employs attention mechanisms for learning in the physical space. Neural Operators based on spectral transformations excel at solving PDEs with smooth solutions but struggle to capture local details, particularly when the solution exhibits sharp variations. Neural Operators based on attention mechanisms exhibit greater adaptability to complex physical phenomena but lack global constraints, resulting in weaker generalization capabilities. In this paper, we propose the Frequency-Domain Corrected Attention Operator (FDCAO), which combines the advantages of both classical Neural Operators. Specifically, the method uses filters to introduce global constraints and enhance the high-frequency response. Then, it enhances the linear attention mechanism through dot-product to further amplify local physical phenomena, thereby better learning rapidly changing complex physical phenomena. Extensive benchmark experiments demonstrate that FDCAO performs excellently across various partial differential equation solving scenarios, effectively learning operator mappings.