CGoFed: Constrained Gradient Optimization Strategy for Federated Class Incremental Learning

Abstract

Federated Class Incremental Learning (FCIL) has emerged as a new paradigm due to its applicability in real-world scenarios. In FCIL, clients continuously generate new data with unseen class labels and do not share local data due to privacy restrictions, and each client’s class distribution evolves dynamically and independently. However, existing work still faces two significant challenges. Firstly, current methods lack a better balance between maintaining sound anti-forgetting effects over old data (stability) and ensuring good adaptability for new tasks (plasticity). Secondly, some FCIL methods overlook that the incremental data will also have a non-identical label distribution, leading to poor performance. This paper proposes CGoFed, which includes relax-constrained gradient update and cross-task gradient regularization modules. The relax-constrained gradient update prevents forgetting the knowledge about old data while quickly adapting to the new data by constraining the gradient update direction to a gradient space that minimizes interference with historical tasks. The cross-task gradient regularization also finds applicable historical models from other clients and trains a personalized global model to address the non-identical label distribution problem. The results demonstrate that the CGoFed performs well in alleviating catastrophic forgetting and improves model performance by 8% -23% compared with the SOTA comparison method.