Performance Comparison of Saliency Detection Methods for Food Region Extraction

Abstract

Several methods for extracting food regions from food images use visual saliency to improve accuracy. The effectiveness of saliency detection methods for food extraction, however, has not been discussed sufficiently. Thus, the effectiveness of well-known saliency detection methods is compared thoroughly for the future development of highly accurate food-extraction methods. Ten saliency detection methods, which consisted of seven handcrafted feature-based approaches and three deep learning-based approaches, were tested by applying them to 240 food images. The results suggest that MSI, which uses only neural networks without the assumption that food regions tend to be found at the center of images, predicted food regions most accurately in terms of areas under a receiver operating characteristic curve (AUC). Additionally, GMR, which assumes that food regions tend not to be found around the four sides of an image, was also effective on the food extraction task. The AUCs of these methods were more than 4% larger than that of a center model that is frequently used as a baseline for saliency detection. Furthermore, this paper supports these results by comparing other methods and determining the properties of food images.