Safeguarding user data privacy in online Large Language Model services

Large Language Models (LLMs), such as GPT, have become central to modern AI applications, including conversational agents, language translation, and document processing. Due to their computational demands, these models are typically hosted on remote servers, requiring users to transmit potentially sensitive data for inference. This raises serious privacy concerns, as user inputs may contain personally identifiable information (PII) and are vulnerable to misuse or unauthorized retention.

To address this challenge, we present PPGPT, a novel and practical privacy-preserving GPT framework. PPGPT employs additive secret sharing to protect user input by enabling secure inference on secret shares rather than raw data. We design secure versions of key transformer components, including GELU and Softmax layers using Beaver’s triples and Taylor series, and introduce an optimized secure layer normalization protocol to reduce overhead.

Experimental results show that PPGPT achieves comparable generation quality to the base model, with a negligible logits difference of $10^{-5}$ and an average inference time of 1.98 s. The framework is lightweight, generalizable to transformer-based LLMs, and suitable for deployment in real-world online services requiring strong privacy guarantees.