Weighted classification of deep and traditional histogram-based features with kernel representation for robust facial expression recognition

Abstract

Feature extraction is crucial in facial expression recognition (FER) systems. This paper introduces a novel descriptor called Local Edge-based Decoded Binary Pattern (LEDB) and a lightweight 1D-CNN named Statistical Local Feature-based Network (SLFNet) to overcome limitations of deep learning approaches, such as the need for complex deep networks, high computational demands, and large training datasets. To enhance feature extraction stability, derivative-Gaussian filters are applied across four directions, yielding more robust representations. In the resulting gradient space, inter-pixel relationships are extracted to generate LEDB micropatterns, which are moderately sized yet highly discriminative, effectively capturing low-level features. Additionally, a compact 1D-CNN with 208k parameters learns high-level features from emotion-related facial regions, enhancing robustness against variations in resolution, noise, and occlusion. High-level and low-level features are fused through a weighted kernel representation strategy to increase resilience to outliers. Extensive experiments on six FER datasets—CK+ , FACES, KDEF, MMI, JAFFE, and RAF-DB—show that the proposed LEDB, SLFNet, and their combination outperform traditional handcrafted descriptors and recent deep learning techniques across various evaluation protocols. Furthermore, the system remains robust in challenging scenarios, such as those with low resolution, noise, or occlusion, which are common hurdles in FER. Code will be made available at: https://github.com/Morteza-Najm/TDF-WKR-FER