Facial Expression Recognition in the Wild: Efficiency of Modified-Category and Ensemble Learning Methods

Abstract

Human facial expression plays a significant role in the medical field, the automotive industry, and so on. Recent research and achievements in recognizing the expressions by using CNNs have been published, conducted on old datasets, i.e. CKP, FER+, etc. However, those are unnatural and less challenging. We authors propose two methods to deal with a new and more realistic dataset called CAER-S first introduced in ICCV 2019. Instead of using the original images of CAER-S, we prepare our dataset by extracting the faces to focus mainly on facial expressions. The first method is to merge some categories sharing the most mispredictions mutually. Nonetheless, the results are indecisive to conclude this method's efficiency. The other is to use plenty of pre-trained models to find the best three of them for ensembles. The ensembles are weighted majority voting and soft voting. These are applied to fuse the three models' results to return the final one whose accuracy is higher than each's separately. This work contributes to advanced facial expression recognition research, especially with using the new dataset CAER-S.