Vertical Federated Representation Synthesis for Non-Aligned Samples in the Active Party

Abstract

In vertical federated learning (VFL) settings, all parties obtain aligned samples of common users identified through private set intersection. For these aligned samples, each party holds disjoint features, while only the active party possesses the label information. As the initiator of model training, the active party can solely benefit from limited aligned samples, overlooking the valuable information contained in its non-aligned local samples. To address the constraint of failing to incorporate the active party's non-aligned samples in practical VFL, this paper presents a Vertical Federated Representation Synthesis (VFedRS) approach for the active party. Its main idea is to transfer the knowledge of common samples to the active party's local feature space, thereby enhancing the performance of the task model trained locally within the active party. Specifically, we first propose a vertical federated principal component analysis module to extract accurate federated representations for common samples under privacy guarantees. To transfer knowledge from federated representations, we then build an input-conditional generative model to synthesize non-aligned samples’ representations for the active party. Subsequently, the active party can leverage data representations to locally train any type of machine learning model suited to the downstream task. We conduct extensive experiments on two tabular datasets and two image datasets in the vertically partitioned setting, and the evaluation results validate the effectiveness of our approach.