Bayesian estimation of the Dirichlet distribution with expectation propagation

As a member of the exponential family, the Dirichlet distribution has its conjugate prior. However, since the posterior distribution is difficult to use in practical problems, Bayesian estimation of the Dirichlet distribution, in general, is not analytically tractable. To derive practically easily used prior and posterior distributions, some approximations are required to approximate both the prior and the posterior distributions so that the conjugate match between the prior and posterior distributions holds and the obtained posterior distribution is easy to be employed. To this end, we approximate the distribution of the parameters in the Dirichlet distribution by a multivariate Gaussian distribution, based on the expectation propagation (EP) framework. The EP-based method captures the correlations among the parameters and provides an easily used prior/posterior distribution. Compared to recently proposed Bayesian estimation based on the variation inference (VI) framework, the EP-based method performs better with a smaller amount of observed data and is more stable.