Unveiling cooperative dynamics in scale-free networks through adaptive deep Q-networks

Abstract

Understanding how cooperation emerges and persists in social dilemmas remains a central focus of scientific inquiry. Previous studies have shown that highly heterogeneous network structures can significantly promote cooperative behavior. To advance this understanding, we investigate the evolution of cooperation in Prisoner’s Dilemma games on scale-free networks and propose a novel strategy update framework: the Adaptive Exploration Deep Q-network Strategy Update method (ADQSU). ADQSU integrates an adaptive exploration mechanism that enables agents to dynamically modulate their exploration probability based on historical payoffs, while employing a Deep Q-Network (DQN) to effectively incorporate past experiences and environmental signals for policy optimization. Additionally, a diversified payoff mechanism is introduced to link individual behavioral states with local social relationships, thereby incentivizing collaborative strategies. Simulation results demonstrate that ADQSU, combined with a neighbor-state-weighted payoff scheme, outperforms traditional update rules in fostering widespread cooperation. Further analysis reveals that increasing the average degree of the network enhances cooperative behavior, with highly connected nodes acting as central propagators of cooperation. These findings not only deepen our understanding of cooperative dynamics in complex networks but also provide actionable insights for the design of cooperation-enhancing mechanisms in decentralized systems.