Semantic-oriented 3D Model Retrieval Using Visual Vocabulary Labelling

Abstract

A novel framework referred to as visual vocabulary labeling is proposed for 3d object retrieval. It aims at localizing the visual semantics to 3d object with textual modalities. Two main processes are included in the presented framework. One is automatic labeling from 3dobject to visual vocabulary. The other is visual vocabulary based retrieval with relevance feedback. The probabilistic model and similarity measurement is proposed to bias the mapping from the low-level 3dobject shape descriptor to the high-level visual vocabularies defined in the visual vocabulary database.The probabilistic model is represented as a cooccurrence matrix of the visual vocabularies. A similarity mapping scheme is devised using statistical methods as Gaussian distribution and Euclidean distance together with probability conditions of visual vocabularies. The proposed 3d object shape descriptor is a 64-dimension vector composed of two kinds of 3dobject features. They are the 19-dimension feature vector from 20 geometry projection images and 45-dimension feature vector from 45 grid region of 3dobject. The technique of relevance feedback has been introduced to the framework both of the labeling process and retrieval process. During the two processes, the proposed method takes the user’s feedback details as the relevant information, and then dynamically updates visual vocabulary database. The experimental results indicate that the proposed semantic-based visual vocabulary descriptors outperform the traditional content-based 3d object retrieval model.