An Instance Segmentation approach to Food Calorie Estimation using Mask R-CNN

Abstract

The aim of this paper is to build a Deep Learning and Computer vision-based model for estimating the calorie contents of any food item (to an extent) using its picture. Deep Learning-based Convolutional Neural Network (CNN) called Mask R-CNN is used to perform the task of instance segmentation. The Mask R-CNN recognizes distinct instances of distinct food objects and outputs a mask for the food objects. The surface area of the detected food item(s) is then computed using the mask. The surface area along with the calorie per square inch value of the food item is used to estimate the calories present in the food. The developed model achieves a mean average precision (mAP) of about 93.7% on food item detection and an accuracy of about 95.5% on calorie estimation.