Combined approach of user specified tags and content-based image annotation

Abstract

The availability of large quantities of user contributed images with labels has provided opportunities to develop automatic tools to tag images to facilitate image search and retrieval. This paper discusses about an approach for automatic annotation in digital images. Some of the previous models for automatic image annotations are translation model (TM), continuous-space relevance model (CRM) and multiple Bernoulli relevance model (MBRM).These models have some semantic gap problems. To avoid these problems here developed a hybrid probabilistic model (HPM) which is used to combine both low-level image features and user provided tags to automatically tag images. For images without any tags, HPM predicts new tags based on the low-level image features. Low-level features are color, texture and shape. For images with user provided tags, HPM use both the image features and the tags to recommend additional tags to label the images. Here a Colored Pattern Appearance Model (CPAM) is used to capture both color and texture information. An L1 norm kernel method is used to estimate the correlations between image features and semantic concepts. The kernel density estimation is accelerated by an Improved Fast Gauss transform(IFGT).When the number of images becomes larger then Tag-Image Association Matrix (TIAM) used in the HPM framework become very sparse, thus it is very difficult to estimate tag-to-tag co-occurrence probabilities. So a collaborative filtering method based on nonnegative matrix factorization (NMF) is used for tackling this data sparsity issue. Here a CF algorithm is used to find the correlation between the words. Building such a HPM will make image labelling more efficient and less labour intensive.