Micro-expression recognition via optical strain and spectral-spatial gating networks

Abstract

Micro-expression recognition (MER) is a challenging task due to the transient nature of facial movements and their susceptibility to interference from rigid head motion and high-frequency sensor noise. Current approaches, which often rely on raw optical flow or purely data-driven architectures, struggle to decouple subtle muscle deformations from this global noise. To address these issues, we propose a Spectral-Spatial-Strain Network (S3Net), a unified physics-aware framework. First, we introduce an early fusion strategy utilizing Dual TV-L1 optical flow and optical strain. By calculating the strain magnitude, we provide the network with a physical prior that helps emphasize non-rigid muscle movements and reduce the influence of head rotations. Second, we design a Spectral Gating Block (SGB) integrated into the ResNet backbone, which leverages the frequency domain to filter sensor noise. Third, a Strain Excitation Block (SEB) is used to recalibrate high-level features in the strain-fused feature stream. Extensive experiments under the official Micro-Expression Grand Challenge 2019 (MEGC2019) 442-sample composite protocol (SAMM, CASME II, and SMIC) demonstrate the effectiveness of our method. S3Net achieves an Unweighted F1-score (UF1) of 85.38% and an Unweighted Average Recall (UAR) of 85.96% on the composite benchmark with the Top-K + flip test-time augmentation (TTA) setting. We also report a strict leave-one-subject-out (LOSO) reference using a fixed stochastic weight averaging (SWA) schedule without held-out-subject checkpoint selection. Together, these results present S3Net under both the final Top-K + flip TTA setting and the stricter no-held-out-selection fixed-SWA reference, while the additional ablation, robustness, efficiency, and statistical analyses clarify the scope and limitations of the method.