Periodic update rule with Q-learning promotes evolution of cooperation in game transition with punishment mechanism

Cooperative behavior assumes a critical role in resolving conflicts arising between collective and individual interests, while punishment measures serve as a robust deterrent against opportunistic free-riding. Within this context, evolutionary game theory (EGT) emerges as an indispensable paradigm for addressing this multifaceted issue. When it comes to introspection behaviors, reinforcement learning (RL) methods exhibit remarkable capabilities to capture agents’ cognitive processes. Nonetheless, previous research has often focused on a static and time-invariant update rule, neglecting the dynamic nature of real-world scenarios where individuals can flexibly transit between strategies in periodic time-dependent patterns. Here, we propose periodic update rules with Q-learning algorithm and game transition model with a punishment mechanism that grants cooperative agents the autonomy to exercise discretion in deciding whether to initiate punishment actions. The agents display dynamic rules periodically through game model transitions, thus ensuring EGT’s inherent adaptability. By employing Monte Carlo (MC) simulations, we analyze the emergence of cooperation that underscores the substantial enhancement of cooperative behavior through the proposed periodic update rules with Q-learning algorithm and game transitions in the presence of punishment. Our study highlights the indispensable significance of appropriate periodic intervals for updating rules and determining optimal punishment costs in the game transition model as critical elements for fostering the evolution of cooperation in real-world scenarios.