On the use of Fourier Features-Physics Informed Neural Networks (FF-PINN) for forward and inverse fluid mechanics problems

Physics Informed Neural Networks (PINN), a deep learning tool, has recently become an effective method for solving inverse Partial Differential Equations (PDEs) where the boundary/initial conditions are not well defined and only noisy sparse measurements sampled in the domain exist. PINN, and other Neural Networks, tends to converge to the low frequency solution in a field that has multiple frequency scales, this is known as spectral bias. For PINN this happens when solving PDEs that exhibit periodic behavior spatially and temporally with multi frequency scales. Previous studies suggested that Fourier Features-Neural Networks (FF-NN) can be used to overcome the spectral bias problem. They proposed the Multi Scale-Spatio Temporal-Fourier Features-Physics Informed Neural Networks (MS-ST-FF-PINN) to overcome the spectral bias problem in PDEs solved by PINN. This has been evaluated on basic PDEs such as Poisson, wave and Gray-Scott equations. In this paper we take MS-ST-FF-PINN a step further by applying it to the incompressible Navier-Stokes equations. Furthermore, a comparative analysis between the PINN and the MS-ST-FF-PINN architectures solution accuracy, the learnt frequency components and the rate of convergence to the correct solution is included. To show this three test cases are shown (a)-Forward time independent double-lid-driven cavity, (b)-Inverse time independent free surface estimation of Kelvin wave pattern, and (c)-Inverse 2D time-dependent turbulent Von Karman vortex shedding interaction downstream of multiple cylinders. The results show that MS-ST-FF-PINN is better at learning low and high frequency components synchronously at early training iterations compared to the PINN architecture that does not learn the high frequency components even after multiple iteration numbers such as the Kelvin wave pattern and the Karman vortex shedding cases. However, for the third test case, the MS-ST-FF-PINN architecture showed a discontinuity for the temporal prediction of the pressure field due to over-fitting.