Handheld Food Localization and Food Recognition Using Convolutional Neural Network

Abstract

In modern society, calories and carbohydrate intake leads to the obesities and diabetes sharply increases. For this reason, food recognition and its application attracted more and more attention. However, a variety of problem such as deformation and color difference cause the difficulty in this task. Especially, localization problem of food item is the most difficult, because the background always colorful and messy. In view of this, optical flow algorithm, which commonly used for foreground separation, is employed in this paper. Based on the speed information, hand-held objects can be isolated from background according to the estimated optical flows. Then, gradient and RGB color value of each pixel in an image are used for recognition. With the advantage of convolutional neural network, high stability and high tolerance, we finally get the remarkable precision in the experiment results, which show the feasibility of our proposed approach for real-world environments.