TFRL-MD: Transformer-Enhanced Federated Reinforcement Learning for Intelligent Microservice Deployment and Resource Allocation

Abstract

Edge computing provides low-latency execution environments for delay-sensitive microservices by deploying computational resources at the network periphery. However, microservice deployment and resource allocation in geographically distributed, heterogeneous edge environments must simultaneously navigate challenges such as data privacy preservation, heterogeneous resource constraints, and dynamic workload fluctuations. Therefore, we propose TFRL-MD, a Transformer-enhanced federated reinforcement learning framework for privacy-preserving and intelligent microservice deployment and resource allocation. The framework models each edge node as an independent Actor-Critic agent to learn deployment policies via local training, enabling cross-node collaborative optimization without sharing raw environmental data. To improve the quality of knowledge fusion among heterogeneous edge nodes, we design a Transformer-based Embedding-Space Aggregation Mechanism (EASM) that leverages multi-head self-attention to adaptively fuse their intermediate-layer knowledge representations. Furthermore, a dual-channel hybrid objective training mechanism is constructed to synchronously inject the aggregated global representations into the optimization targets of both Actor and Critic networks as differentiable semantic embeddings, thereby smoothly guiding local gradient descent and eliminating the policy oscillations inherent in traditional parameter overwriting.Dynamic simulation experiments, conducted using the EUA dataset from the Australian Central Business District (CBD) region, demonstrate that TFRL-MD outperforms various baselines in both average response time and load balancing, achieving maximum improvements of 46.01% and 61.11%, respectively.