Bayesian Model Selection for Independent Factor Analysis

Abstract

We present a stochastic algorithm for Independent Factor Analysis, incorporating a scheme for performing model selection over latent data. Independent Factor Analysis (IFA) is a method for learing locally non-linear subspaces in data. IFA uses a hierarchical generative model with factors modeled as independent Mixtures of Gaussians(MoGs), each mixture component representing a factor state. We incorporate Birth-Death MCMC (BDMCMC) to simulate samples from the posterior distribution of the factor model, with a Gibbs Sampler simulating from the posterior over model parameters. In spite of the common practice of using a fixed number of mixture components to model factors, it may be difficult to blindly determine an optimal minimal number of components without prior knowledge of the structure of the hidden data. Also, in pattern recognition applications where the source model order has an intrinsic interpretation, estimating this along with other model parameters would be useful. Our algorithm addresses both issues of model selection and parameter estimation.