Semantics Sensitive Segmentation and Annotation of Natural Images

Abstract

In this paper, we present new perceptual techniques for segmentation and annotation of natural images. The image segmentation approach is a multilevel clustering method based on a new proposed non-parametric clustering algorithm, called adaptive medoidshift (AMS) and normalized cuts (N-cut). The AMS method locally clusters the image color composition by considering their spatial distribution into uniform segments, which are then perceptually group together using N-cut into meaningful semantic sensitive salient regions. The proposed image annotation approach assigns labels at segment and scene level to represent semantic content and concept of image respectively. The low level features are extracted from the obtained salient regions and are used by support vector machine (SVM) classifiers to assign segment labels, which are then used to derive scene labels. This effectively reduces the ¿semantic gap¿ between low level features and high level semantics. Experiments show the effectiveness of proposed algorithms on variety of natural images.