THE-TAFL: Transforming Healthcare Edge with Transformer-based Adaptive Federated Learning and Learning Rate Optimization

Abstract

The healthcare industry is becoming more vulnerable to privacy violations and cybercrime due to the pervasive dissemination and sensitivity of medical data. Advanced data security systems are needed to protect privacy, data integrity, and dependability as confidentiality breaches increase across industries. Decentralized healthcare networks face challenges in feature extraction during local training, hindering effective federated averaging and learning rate optimization, which affects data processing and model training efficiency. This paper introduces a novel approach of Transforming Healthcare Edge with Transformer-based Adaptive Federated Learning (THE-TAFL) and Learning Rate Optimization. In this paper, we combine Transformer-based Adaptive Federated Learning (TAFL) with learning rate optimization to improve the privacy and security of healthcare information on edge devices. We used data augmentation approaches that generate robust and generalized input datasets for deep learning models. Next, we use the Vision Transformer (ViT) model for local training, generating Local Model Weights (LMUs) that enhance feature extraction and learning. We designed a training optimization method that improves model performance and stability by combining a loss function with weight decay for regularization, learning rate scheduling, and gradient clipping. This ensures effective training across decentralized clients in a Federated Learning (FL) framework. The FL server receives LMUs from many clients and aggregates them. The aggregation procedure utilizes adaptive federated averaging to aggregate the LMUs based on the performance of each client. This adaptive method ensures that high-performing clients contribute more to the Global Model Update (GMU). Following aggregation, clients receive the GMU to continue training with the updated parameters, ensuring collaborative and dynamic learning. The proposed method provides better performance on two standard datasets using various numbers of clients.