Learnable difference convolution and temporal feature supplementary module for near-infrared eye segmentation

Abstract

Near-infrared (NIR) eye image segmentation refers to a pixel-wise semantic segmentation technique for ocular images captured in close-range NIR imaging scenarios. Serving as a prerequisite for augmented reality (AR), virtual reality (VR) eye-tracking systems, and biometric technologies, the segmentation accuracy of this technique directly determines the overall performance of downstream tasks. However, accurately segmenting ocular structures under unconstrained conditions remains challenging, primarily due to feature ambiguity caused by light reflection artifacts, interferences from eyelashes and eyelids, and temporal inconsistency induced by frame-by-frame independent segmentation. To address the above issues, this paper proposes a learnable difference convolution (LDC) and a feature supplementary module (FSM) that introduces temporal priors via feature-level fusion within skip connections, employing foreground guidance and background compensation. The LDC does not rely on fixed-pattern difference convolution kernels; instead, it learns task-optimal difference convolution kernels during the training process, thereby enhancing the model’s boundary feature extraction capability and its robustness against occlusions and reflection interference. Meanwhile, the FSM leverages the segmentation result of the previous frame and the ocular difference map as temporal contextual priors to guide the segmentation process of the current frame through feature-level fusion and foreground/background compensation, which mitigates the problem of temporal inconsistency. Evaluations across three benchmark datasets demonstrate that our method outperforms state-of-the-art approaches in eye image segmentation tasks. The code is publicly available at https://github.com/qinhaiyu218/LDC-FSM-Eye-Segmentation.