Estimation information bounds using the I-MMSE formula and Gaussian mixture models

Abstract

We derive a method to bound the mutual information between a noisy and noiseless measurement exploiting the I-MMSE estimation and information theory connection. Modeling the source distribution as a Gaussian mixture model, a closed form expression for upper and lower bounds of the minimum mean square error is found using recent results. Using the connection between rate of information relative to SNR and the minimum mean square error of the estimator, the mutual information can be bounded as well for arbitrary source distributions in Gaussian noise.