Uncertainty Prediction for Facial Action Units Recognition under Degraded Conditions

Facial action units (AUs) represent muscular activities, and their recognition from facial images can capture various psychological states, such as people’s interests as consumers and mental health states. However, degradation of conditions, such as occlusions by hand, often occurs and affects the accuracy of AUs recognition in the real world. Most existing studies on degraded conditions have adopted the approach using additional training images and advanced structures of neural networks to improve the robustness of AUs recognition from a degraded facial image. However, such an approach cannot deal with cases in which evidence of the AUs is completely or almost invisible. Therefore, we propose a novel method to address the degraded conditions by predicting the uncertainties of the AUs recognition caused by them. Our method interpolates the high-uncertainty data using surrounding data to reduce the influence of the degraded conditions, and visualizes the conditions causing the uncertainties to handle cases where the conditions are very poor and need to be improved. In the evaluation experiments, the public datasets BP4D+ and DISFA were modified to degrade them for testing. By evaluating the modified test data, we demonstrated that the maximum improvement with our method was 12% for BP4D+ and 17% for DISFA, and that our method can prevent the decrease in accuracy owing to degraded conditions. We also presented some visualization examples which demonstrate that our method can reasonably predict the conditions and uncertainties.