A lightweight finger multimodal recognition model based on detail optimization and perceptual compensation embedding

Multimodal biometric recognition technology has attracted the attention of many scholars due to its higher security and stability than single-modal recognition, but its additional parameter quantity and computational cost have brought challenges to the lightweight deployment of the model. In order to meet the needs of a wider range of application scenarios, this paper proposes a lightweight model DPNet using fingerprint and finger vein images for multimodal recognition, which adopts a double-branch lightweight feature extraction structure combining detail optimization and perception compensation. Among them, the detail extraction optimization branch uses multi-scale dimensionality reduction filtering to obtain low-redundant detail information, and combines the depth extension operation to enhance the generalization ability of detail features. The perception compensation branch expands and compensates the model's perceptual field of view through lightweight spatial location query and global information attention. In addition, this paper designs a perceptual feature embedding method to embed perceptual compensation information in the way of importance adjustment to improve the consistency of embedded features. The ABFM fusion module is proposed to carry out multi-level lightweight and deep interactive fusion of the extracted finger modal features from the global to the spatial region, so as to improve the degree and utilization rate of feature fusion. In this paper, the model recognition performance and lightweight advantages are verified on three multimodal datasets. Experimental results show that the proposed model achieves the most advanced lightweight effect and recognition performance in the experimental comparison of all datasets.