Food Recognition: Can Deep Learning or Bag-of-Words Match Humans?

Abstract

: Automated smartphone-based food recognition is a useful basis for applications targeted at dietary assessment. Dish recognition is a necessary step in that process. One of the possible approaches to use is deep learning-based recognition, another one is bag-of-words based classification. Deep learning has increasingly become the preferred approach to use in either this or other image classification tasks. Additionally, if humans are better recognizing the dish, the automated approach is useless (it will be less error-prone for the user to identify the dish instead of capturing the photo). We compare the alternatives of Deep Learning (DL), Bag-of-words (BoW) and Humans (H). The best deep learner beats humans when on few food categories, but looses if it has to learn many more food categories, which is expected in real contexts. We describe the approaches, analyze the results, draw conclusions and design further work to evaluate further and improve the approaches.