Secure Embedding Aggregation for Cross-Silo Federated Representation Learning

Abstract

Representation learning plays a pivotal role in modern applications by enabling high-quality embeddings that support various downstream tasks such as recommendation, clustering, and personalized services. In federated representation learning (FRL), a central server collaborates with N clients, each holding private data, to jointly learn representations of entities (e.g., users in a social network). However, existing embedding aggregation protocols often fall short in either ensuring privacy protections or fully leveraging aggregation opportunities, leaving sensitive data exposed or vulnerable to collusion. To address these challenges, we propose SecEA, a secure embedding aggregation protocol that fully exploits all potential aggregation opportunities across all entities among clients while providing provable privacy guarantees. SecEA defends both local entities and their embeddings—ensuring computational security against a curious server and statistical privacy against up to $T \lt N/2$ colluding clients. Comprehensive experiments on various representation learning tasks in cross-silo scenarios demonstrate that SecEA incurs a negligible performance loss (within 5%) compared to protocols with weaker or no privacy guarantees, and its additional computational latency significantly diminishes when training deeper models on larger datasets. A parallel mechanism is also included, which helps further improve the efficiency linearly. These results underscore that SecEA not only provides full privacy protections for both entity and embedding, but also preserves the utility of the learned representations.