FedBH: Efficient federated learning through shared base and adaptive hyperparameters in heterogeneous systems

Federated learning is a distributed learning framework that protects client data privacy. However, in real-world applications, due to the significant differences in data distribution among different clients, a single global model often fails to meet personalized needs. Moreover, existing personalized federated learning methods often struggle to balance computational efficiency and model personalization while inadequately addressing the heterogeneous computational capabilities of clients. To address these challenges, we propose FedBH—an innovative personalized federated learning method that combines the feature extraction capabilities of deep learning with an adaptive dynamic training mechanism. FedBH decomposes each client’s model into a global base layer and client-specific local heads, enhancing both computational efficiency and personalized training. To further optimize learning, FedBH incorporates a dynamic adjustment mechanism that adapts training parameters based on each client’s specific conditions. In each local training round, the algorithm samples a subset of the dataset and explores different configurations of local head and global base training rounds. The optimal configuration is determined based on validation loss and then applied for full training. This adaptive mechanism enables FedBH to dynamically adjust to varying client demands, ensuring efficient utilization of computational resources while maintaining robust personalization. Experimental results across multiple benchmark datasets and diverse data distribution scenarios show that FedBH achieves faster convergence and higher accuracy in most cases. These findings demonstrate that FedBH not only effectively adapts to heterogeneous data environments but also significantly improves training efficiency, showing great potential in addressing issues related to diverse and non-IID data distributions.