Stochastic complexity of complete bipartite graph-type Boltzmann machines in mean field approximation

Abstract

A variational Bayes method is proposed as an approximation method for implementing Bayesian posterior distribution with moderate computational complexity. Its effectiveness for real problems has been confirmed. The variational Bayes method is a method which generalizes the mean field approximation used in calculating partition functions in statistical physics. In recent years, the mathematical properties of the precision with which it is approximated have been investigated. In this paper, the authors consider the asymptotic stochastic complexity in the case of applying the mean field approximation to complete bipartite graph-type Boltzmann machines and theoretically derive that asymptotic form. Also, based on the results, the authors quantitatively consider the difference between Bayesian posterior distribution and the posterior distribution of the mean field approximation. © 2007 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 90(9): 1– 9, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20307