Evolving Gaussian Systems as a Framework for Federated Regression Problems

Abstract

In this article, we present a novel federated learning framework to multivariate regression problems, termed evolving Gaussian federated regression (eGauss+$_{\text{FR}}$). The need for a federated approach is due to the increasing problem of distributed acquisition of the data and protection for the rights of distributing these data. Regression problems are usually nonlinear and, therefore, strongly connected to the clustering to divide the data space into smaller subspaces where a linear approximation could be applied. Here, we are faced with the main drawback of traditional clustering methods, where a predefined number of clusters are needed. In federated learning problems, where the data are commonly nonidentically distributed between different sources or clients, this represents a significant challenge. This problem can be overcome by introducing an evolving approach, which adds and removes the clusters on-the-fly. The idea in our approach is to use the incremental c-regression or c-varieties clustering methods to define the clusters, which lie close to the lines and describe them with the centers and the covariance matrices. The clustering is done for each data source or client. Due to the restriction and protection of data sharing, only the centers and the covariance matrices of all clients are then transmitted to main server and merged together, which is here done in a way as proposed in eGauss+ method. From merged clusters the auxiliary points are generated, which than serve to approximate the function by using classical fuzzy models. Our proposed method was demonstrated on simple synthetic data, while synthetic and real-world datasets were used to test time complexity and scalability with the number of clients. The results demonstrate the benefits of evolving federated method, which results in high-quality approximation of the function and can be easily extended to high-dimensional problems.