Are subtle expressions too sparse to recognize?

Abstract

As subtle emotions are slightly and involuntarily expressed, they need to be recorded by high-speed camera. Though this high frame-per-second rate allows better capture of subtle expressions, it typically generates a lot of redundant frames with rapid varying illumination and noise but without significant motions. The redundancy is analyzed and eliminated by Sparsity-Promoting Dynamic Mode Decomposition (DMDSP), which helps synthesize dynamically condensed sequences. Moreover, DMDSP can also visualize dynamics of subtle expressions in both temporal and spectral domains. As meaningful subtle expressions are temporarily sparse, DMDSP would be able to extract these meaningful dynamics and improve recognition rates of subtle expressions. The hypothesis is evaluated on CASME II, a database of spontaneous subtle facial expressions. Recognition performance measured by F1-score, recall and precision metrics showed a significant leap of improvement when DMDSP is used to preserve a small percentage of meaningful frames in sequences with temporally high sparsity levels.