A new multimodal fusion method based on association rules mining for image retrieval

Abstract

The retrieving method proposed in this paper utilizes the fusion of the images' multimodal information (textual and visual) which is a recent trend in image retrieval researches. It combines two different data mining techniques to retrieve semantically related images: clustering and association rules mining algorithm. The semantic association rules mining is constructed at the offline phase where the association rules are discovered between the text semantic clusters and the visual clusters of the images to use it later at the online phase. The experiment was conducted on more than 54,500 images of ImageCLEF 2011 Wikipedia collection. It was compared to an online image retrieving system called MMRetrieval and to the proposed system but without using association rules. The obtained results show that the proposed method achieved the best precision score among different query categories.