Third-party private set intersection with application to privacy-preserving training of large language models

Abstract

In the training of large language models (LLMs), the protection of private dataset is especially crucial. The private set intersection (PSI) mechanism acts as a potent privacy-preserving collaborative learning technique, allowing participants to collaborate in model training without revealing their own data, and thereby meeting the training requirements of LLMs. In this paper, we consider a variant of PSI, namely third-party PSI, where a third-party with no input privately receives the intersection of the other two parties’ sets, while the two parties output nothing. We propose a general construction of third-party PSI protocol from leveled fully homomorphic encryption, which ensures privacy-preserving training of large language models. The proposed construction can support intersection of arbitrary-length items by using polynomial links, and its security can be proven in the presence of semi-honest adversaries. Compared with existing protocols, the instantiation of the proposed general construction achieves higher computational efficiency while maintaining equivalent level of communication complexity. More importantly, the proposed protocol offers better utility, effectively safeguarding the privacy of the data without compromising model accuracy.