Combining eye movements for semantic image classification

Abstract

Nowadays, the “semantic gap” problems have greatly limited development of image classification. The key to this problem is to get semantic information of the images. A semantic image feature extraction method is proposed in this paper, in which eye movement information is integrated. Firstly, the underlying visual features of images are extracted. Secondly, weighed feature vectors of images are constructed based on eye movements and underlying visual features. To evaluate the effectiveness of the integrated feature vectors in classification, both support vector machine and k - nearest neighbor algorithm are adopted. Experimental results demonstrate the effectiveness and efficiency of the proposed methods.