Vertin: Fast, Communication-friendly and Key-compact secure inference system for NNs and LLMs

Existing secure inference schemes based on function secret sharing (FSS) allow the client to obtain inference results while protecting the client’s inputs, the server’s neural networks (NNs), and large language models (LLMs), ensuring high online efficiency. However, there is still room for improvement in terms of storage, communication, and inference speed for linear layers in these schemes. In this work, we introduce a novel semi-honest secure two-party inference system tailored for NNs and LLMs, which surpasses state-of-the-art solutions in speed, communication efficiency, and key storage. Our system leverages plaintext weight matrices for the server, introducing FMLO, a secure two-party computation protocol supporting linear operations. By using precomputed random matrices correlated with weight matrices, FMLO minimizes key storage, online computation, and communication demands. We also develop two efficient protocols, ${\pi }_{MulPre}$ for matrix multiplication and ${\pi }_{ConvPre}$ for matrix convolution, by using vector oblivious linear evaluation. Both protocols batch-generate required random numbers securely in the offline phase, reducing preprocessing overhead in FMLO. Compared to the leading FSS-based scheme Orca, Vertin reduces key storage by 5.37%, online communication by 16.46%, and online inference time by 10.71% in secure inference with ResNet-50. When compared to the state-of-the-art SIGMA on BERT-large model with the sequence length of 64, Vertin achieves reductions in key storage, online communication, and online runtime by 9.81%, 9.17%, and 8.9% respectively.