Mutual Retrieval of Pictures and Sounds on the Web Based on Impression

Abstract

This paper shows the system mutually retrieving pictures and sound clips based on their impressions. Pictures (sound clips, respectively) are retrieved by specifying a music clip (picture). Pictures (sound clips) having similar impression to a sound clip (picture) are retrieved. No words are required in retrieving them. To this end, the scores of the factors representing the impression of pictures and sound clips are used. This paper shows the estimation methods of the scores of the factors of pictures and sound clips. The factor scores estimated from the feature values of pictures and sound clips by using the estimation formulas are used in the retrieval. It is shown that the prototype system could mutually retrieve pictures and sound clips on the web successfully.