Content-based image retrieval based on eye-tracking

Abstract

To improve the performance of an image retrieval system, a novel content-based image retrieval (CBIR) framework with eye tracking data based on an implicit relevance feedback mechanism is proposed in this paper. Our proposed framework consists of three components: feature extraction and selection, visual retrieval, and relevance feedback. First, by using the quantum genetic algorithm and the principle component analysis algorithm, optimal image features with 70 components are extracted. Second, a finer retrieving procedure based on multiclass support vector machine (SVM) and fuzzy c-mean (FCM) algorithm is implemented for retrieving most relevant images. Finally, a deep neural network is trained to exploit the information of the user regarding the relevance of the returned images. This information is then employed to update the retrieving point for a new round retrieval. Experiments on two databases (Corel and Caltech) show that the performance of CBIR can be significantly improved by using our proposed framework.