Explainable neural-networked variational inference: A new and fast paradigm with automatic differentiation for high-dimensional Bayesian inverse problems

Abstract

Bayesian inference offers a rigorous framework for parameter inversion and uncertainty quantification in engineering disciplines. Despite advancements introduced by Variational Bayesian Inference (VBI), Bayesian Inverse Problems (BIPs) with implicit and non-differentiable forward solvers still face significant limitations associated with mean-field approximation, computational difficulties, poor scalability, and high-dimensional data complexities. In response to these challenges, a novel Variational Inference (VI) framework featuring equivalent neural network representation with automatic differentiation is proposed. The network architecture “VBI-Net”, comprising a variational distribution sampler, a likelihood function approximator, and a variational free energy loss function, is designed to mirror the VI framework with multivariate Gaussian variational distributions. The sampler yields posterior samples of the system model parameters and prediction errors, while incorporating the variational parameters as differentiable and explainable network parameters by reparameterization trick. The likelihood function approximator employs a neural network as a viable replacement for time-intensive and non-differentiable forward solvers, enabling efficient likelihood function evaluations. The loss function measures the goodness of the variational distribution. The seamless integration of the sampler and approximator guarantees the overall differentiability of the architecture, facilitating the utilization of automatic differentiation, gradient-based optimization methods, and enabling scalability to high-dimensional scenarios. Furthermore, the explainable neural-networked implementation scheme leverages CUDA support embedded in deep learning frameworks to inherently enable parallel computation, GPU acceleration, and optimized tensor operations. To demonstrate its efficacy, the method is applied in Bayesian model updating scenarios involving a numerical shear building and a practical structure.