Physics-informed neural network model using natural gradient descent with Dirichlet distribution

Abstract

In this article we propose the physics-informed neural network model which contains the natural gradient descent with Dirichlet distribution. Such an optimizer can more accurately converge in the global minimum of the loss function in a short number of iterations. Due to natural gradient, one considers not only the gradient directions but also convexity of the loss function. Using the Dirichlet distribution, natural gradient allows for a reduction in time consumption comparing with the second order approaches. The proposed physics-informed neural model increases the accuracy of solving initial and boundary value problems for partial differential equations, such as the heat and Burgers equation, on $0\text{\%}-10\text{\%}$ Gaussian noised data. Compared with the state-of-the-art optimization methods, the proposed natural gradient descent with Dirichlet distribution achieves the more accurate solution by $9\text{\%}-62\text{\%}$, estimated by mean squared error and $L^2$ error.