An improved end-to-end micro-expression recognition system for real-world videos via dual-input CNN architecture

Micro-expression (ME) recognition reveals nonverbal emotions through subtle, involuntary facial muscle movements. However, the development and commercialization of ME recognition systems have been hindered by the lack of databases that accurately reflect real-world conditions. This study addresses this challenge by proposing a robust end-to-end system designed to operate effectively in unconstrained environments. Existing methods typically rely on a single apex frame, which may be unreliable due to noise, occlusions, or lighting variations. To address these issues, a 3D facial reconstruction technique is applied as a pre-processing step to normalize pose and lighting. A novel dual-peak frame detection strategy is then introduced to extract two expressive optical flow frames, reducing the impact of noise from any single frame. Finally, a Shallow and Small-size Dual-input (SSD) CNN architecture is designed to jointly process the two frames for improved emotion classification. The proposed system achieves strong performance on the challenging in-the-wild MEVIEW dataset, with accuracy and F1-score of 75 % and 77.68 %, respectively. Comprehensive evaluations further validate the effectiveness of the pipeline, highlighting its potential for real-world ME recognition applications.