Multi type mean field reinforcement learning for optimal resource allocation in heterogeneous network

With the exponential growth in the amount of data transmitted over mobile networks, contemporary 5G communication technologies with the primary goal of improving network performance and quality of service have gained much attention. Efficient resource allocation and interference management are especially critical in large-scale wireless networks. Device-to-device (D2D) communication has become a promising technological tool to address this growing need. However, the limitation of exponentially growing solution space in large-scale ultra-dense networks makes it difficult to achieve real-time control with conventional optimization methods. To face this challenge, we propose a novel framework that combines Multi-Agent Reinforcement Learning (MARL) with Mean Field Type Game (MFTG) theory, allowing agents to operate in different action spaces. This approach extends the core principle of mean-field reinforcement learning from a single type to multiple types of interactions, effectively modeling the approximate behavior between various types of devices in heterogeneous D2D networks. Experimental results show that the proposed Multi-Type Mean-Field double deep Q-network (MTMF-Q) method outperforms benchmark methods in heterogeneous networks. In addition, the proposed method exhibits good scalability in parameters such as user density, network size and power budget, showing its potential for application in ultra-dense heterogeneous communication network scenarios.