A visual search system for video and image databases

This paper describes a new methodology for image search that is applicable to both image libraries and keyframe search over video libraries. In contrast to previous approaches, which require templates or direct manipulation of low-level image parameters, our search system classifies images into a pre-defined subject lexicon, including terms such as trees and flesh tones. The classification is performed off-line using neural network algorithms. Query satisfaction is performed on-line using only the image tags. Because most of the work is done off-line, this methodology answers queries much more quickly than techniques that require direct manipulation of images to answer the query. We also believe that pre-defined subjects are easier for users to understand when searching programmatic video material. Experiments using keyframes extracted by our video library system show that the methodology gives high-quality query results with fast on-line performance.