BiModal latent dirichlet allocation for text and image

Abstract

A BiModal Latent Dirichlet Allocation Model(BM-LDA) is proposed to learn a unified representation of data that comes from both the textual and visual modalities together. The model is able to form a unified representation that mixs both the textual and visual modalities. Based on the assumption, that the images and its surrounding text share a same topic, the model learns a posterior probability density in the space of latent variable of topics that bridging over the observed multi modality inputs. It maps the high dimensional space consist of the observed variables from both modalities to a low dimensional space of topcis. Experimental result on ImageCLEF data set, which consists of bi-modality data of images and surrounding text, shows our new BM-LDA model can get a fine representation for the multi-modality data, which is useful for tasks such as retrieval and classification.