FedHarmony: A Unified Framework for Addressing Clinical Heterogeneity in Federated Medical Image Analysis

Abstract

Federated Learning (FL) offers a privacy-preserving approach for collaborative modeling of federated medical imaging data across multiple centers. However, clinical heterogeneity, which arises from differences in imaging equipment, scanning protocols, and patient populations across centers, leads to performance drift and limits the model’s global generalization. Existing methods typically optimize client selection and model aggregation separately, which makes it difficult to mitigate the combined negative effects of unrepresentative training queues and unstable model fusion. In this study, we propose FedHarmony, a unified framework designed to address clinical heterogeneity in federated medical image analysis. The focus of this work is heterogeneity-aware federated learning for multi-center medical imaging, where variations in imaging quality, acquisition conditions, and patient distributions may jointly impair robustness, fairness, and generalization. Specifically, we introduce two novel mechanisms: Proactive Client Orchestration (PCO) and Stability-aware Model Fusion (SMF). PCO strategically guides training by evaluating each client’s potential performance gains and gradient complementarity, thereby improving overall effectiveness. SMF enhances robustness by introducing noise perturbations during local training to promote flatter minima, and it assigns aggregation weights according to model stability, thus improving both generalization and training stability. Experimental results show that FedHarmony outperforms existing state-of-the-art methods in accuracy and stability across diverse medical imaging tasks, while effectively balancing communication costs and model performance. Ablation studies further confirm the complementary benefits of PCO and SMF.