Food Image Categorization Using Attentional Bilinear Model

Abstract

Nowadays, many food images are posted on various social network platforms without identification labels. An automatic food categorization application would greatly help to identify and classify food categories. Food categorization is a complex problem since the number of category types can be more than one hundred. Many kinds of food are similar with only subtle differences in taste and presentation and this can lead to a problem called “finegrained issue”. Recently, a bilinear model was employed which showed good accuracy and generated excessive features to capture details among different food categories, albeit with limited performance. Diverse food categories require disparate sets of features. Here, an attention mechanism was applied to capture suitable features and specifically identify each food category. Furthermore, the performance of a bilinear backbone was also enhanced by applying Inception in correlation with Inception-ResNet-v2 and Inception-v3 networks. The experiment was conducted on the Wongnai dataset containing various images that were separated into 83 classes. Results showed that our attentional model outperformed the traditional bilinear model, with an average of 16% improvement showing 3% and 44% as min-max performance values, respectively.