Ophiuchus: Privacy-preserving training service with user-controlled pseudo-noise information generation

Cloud-based learning services face the risk of privacy information leakage. Thus, many cryptography-based inference schemes have been proposed. However, the requirement for protecting backpropagation and gradient updates makes training much more complex and costly than inference. To fill the gap between inference and training, we optimize the amount of cryptographic computation during the backpropagation. Specifically, we demonstrate that parameter gradients are separable, thus extending an existing private inference scheme to private training. Furthermore, we pioneer the integration of normalizer-free learning into private training, circumventing the normalization layers, which are cryptography-unfriendly. Moreover, to defend against reconstruction attacks, we construct pseudo-noise information by introducing contrastive loss, which is based on the confidentiality of labels and the randomness of positive–negative pairs. Putting it all together, we propose Ophiuchus, a private CNN training framework for image recognition. Empirical results on several benchmark datasets demonstrate that Ophiuchus achieves accuracy comparable to plain training and the backpropagation only incurs an additional overhead of $2.3\text{\%–}5.6\text{\%}$. Our scheme can improve the performance between $1.6\times \text{–}4.1\times$ compared to the current private training schemes. Notably, the constructed pseudo-noise outperforms random noise in both aspects of privacy and utility.