PrivLSTM: A privacy-preserving LSTM inference framework by fusing encryption and network structure for multi-sourced Data

Machine Learning as a Service (MLaaS) has emerged as a prominent topic for dealing with multi-sourced data. However, the privacy concerns have received more considerable attention than ever. In this paper, we propose PrivLSTM, a low latency privacy-preserving long short-term memory (LSTM) neural network inference method by fusing the encrypted data and network structure, protecting the sequence data input to the LSTM. PrivLSTM is implemented with a non-interactive framework using fully homomorphic encryption and is adaptive to a general LSTM structure with long input sequences. PrivLSTM uses bootstrapping to avoid the impact of noise introduced by the deep multiplications on the accuracy. The computation overhead of bootstrapping is alleviated by developing a novel batch-based linear transformation method, which reduces the automorphism operations and integrates identical operations in the LSTM cell. The nonlinear operations, e.g., sigmoid and tanh, are approximated with optimized polynomials. Besides, the efficiency and security of PrivLSTM are theoretically analyzed. We conduct experiments on multi-sourced datasets, such as text and sensory datasets. The results show that PrivLSTM achieves comparable accuracy against the plain LSTM and fast computation with around 30s for a 100-length input sequence.