Privacy-preserving face recognition using a cryptographic end-to-end optoelectronic hybrid neural network

Abstract

Privacy-preserving technology has emerged in recent years as a solution to the privacy leakage issues associated with biometric recognition. However, the task of recognizing individuals in large datasets presents formidable challenges, including high computational burden and time delays, while still ensuring privacy protection and efficiency. To tackle these challenges, we proposed a novel cryptographic end-to-end optoelectronic hybrid neural network (CE²OHNN) designed specifically for privacy-preserving face recognition. The CE²OHNN cascades a compact optical encryption (COE) system at the front-end for image encryption, and an electronic neural network (ENN) at the back-end for face recognition directly on ciphertext images. This architecture not only enables real-time inference by utilizing high-speed and parallel optical encryption instead of digital encryption, but also offers enhanced security and computational resource savings by eliminating the need for image decryption. In this work, we proposed a compact and lightweight COE system, the security of which is validated against known-plaintext attack (KPA). Through simulations with the ORL dataset, we assess the recognition accuracy of the CE²OHNN, which achieves an accuracy of 96.67%, comparable to the baseline model’s 99.17%. Furthermore, this work holds potential for extension to other applications such as recognition of de-identified attributes like age and gender.