Optimizing CPU-GPU resource scheduling with deep reinforcement learning

Abstract

The rapid development of the Internet of Things (IoT) and Edge Computing has created a greater need for real-time and accurate information updates in latency-sensitive applications. We address heterogeneous devices, including CPU-only, GPU-only, and hybrid CPU-GPU devices, by constructing a Markov Decision Process (MDP) model that effectively captures the dynamic characteristics of these devices and Edge Server (ES). To fully leverage the heterogeneity of wireless devices(WDs), we propose a Deep Reinforcement Learning (DRL) algorithm based on a Multi-Head Attention, where each device type is assigned independent attention weights to capture its impact on task scheduling decisions efficiently. To address the slow convergence issue in traditional Reinforcement Learning (RL) algorithms under completely unknown systems, we propose heterogeneous computing-aware Post-Decision States (PDS) learning. This mechanism incorporates partial prior knowledge of the dynamics of the system in edge environments to accelerate the exploration and learning process. Experimental results demonstrate that the proposed method significantly optimizes both the Age of Information (AoI) and the energy consumption performance in heterogeneous edge environments, outperforming existing approaches.