LLM4Game: Multi-agent reinforcement learning with knowledge injection for dynamic defense resource allocation in cloud storage

Abstract

The non-cooperative and interdependent nature of network attack-defense links it closely to game theory. Current game-theoretic decision-making methods construct game models for attack-defense scenarios and use reinforcement learning (RL) to compute optimal strategies. However, RL relies on the “trial and error” exploration and is likely to fall into the local optimum in some cloud storage environment without game equilibrium. First, in cloud storage systems, the resource investment of attack and defense players has a “winner-takes-all” characteristic. Thus, we employ the Colonel Blotto game to model the attack-defense scenario in cloud storage systems, extending it to a multi-player, heterogeneous battlefield model with asymmetric resources. Second, RL’s reliance on trial-and-error exploration leads to suboptimal convergence in sparse-reward, non-equilibrium conditions. We leverage Large Language Models (LLMs) to inject attack-defense context knowledge, addressing the cold start problem of RL. Finally, we propose the RL-LLM-KI algorithm featuring a precomputation-retrieval mechanism that mitigates the inference speed discrepancy between LLMs and RL agents, enabling real-time defense decisions. Experiments show that our work increases utility by 140 % and 136.36 % compared to MADRL and DRS-DQN respectively in typical experimental scenarios. To our best knowledge, this study is the first to reveal the significant effect of knowledge injection in enhancing decision-making efficacy in highly adversarial cloud storage attack-defense scenarios.