Identity Reflections: Hyperspectral Skin Analysis via Reflectance Distance Learning With Context-Aware Contrastive Transformers

Abstract

Hyperspectral Imaging (HSI) enables real-time analysis with rich spectral detail, offering a promising path to spoof-resistant AI-based identity verification. Leveraging this spectral information, we present a novel biometric approach that extracts and analyzes skin reflectance patterns from hyperspectral fingertip images. In prior work, hyperspectral analysis has primarily been applied to facial biometrics; however, the associated lighting requirements are often impractical or inconvenient for users in real-world scenarios. From a computational perspective, while deep learning has significantly advanced recognition using conventional imaging, effectively applying these frameworks to the rich and complex signals captured by HSI remains a critical open challenge. Convolutional Neural Networks (CNNs) often have difficulty preserving the sequential structure of spectral information. Vision transformers effectively capture global context through self-attention but often face challenges in modeling fine-grained local features due to the lack of convolutional inductive biases. To address existing limitations, this work introduces a novel integration of Context-Enriched Contrastive Loss (CECL) into two architectures: SpectralFormer, a ViT-based model that employs decoupled spatial–spectral processing to handle spectral signatures extracted from carefully selected spatial regions; and the Swin Transformer, which leverages a unified 3D joint representation. For the first time, SpectralFormer is employed for identity verification, leveraging group-wise spectral embeddings, inspired by ViT’s patch-based processing, across adjacent bands, along with intermediate skip connections to enhance the analysis of complex spectral data. The proposed approach was evaluated on three biometric databases, including the Mason FHSD-2022 hyperspectral fingertip dataset collected at George Mason University with 100 subjects. The code will be publicly available on GitHub.