Non-cooperative Learning for Robust Spectrum Sharing in Connected Vehicles with Malicious Agents

Abstract

Multi-agent reinforcement learning (MARL) has pre-viously been employed for efficient spectrum sharing among co-operative connected vehicles. However, we show in this paper that existing MARL models are not robust against non-cooperative or malicious agents (vehicles) whose spectrum selection strategy may cause congestion and reduce the spectrum utilization. For example, a selfish (non-cooperative) agent aims to only maximize its own spectrum utilization, irrespective of the overall system efficiency and spectrum availability to others. We investigate and analyze the MARL-based spectrum sharing problem in connected vehicles including vehicles (agents) with selfish or sabotage strategies. We then develop a theoretical framework to consider the selfish agent, and study various adversarial scenarios (including attacks with disruptive goals) via simulations. Our robust MARL approach where “robust” agents are trained to be prepared for selfish agents in testing phase achieves more resiliency in the presence of a selfish agent and even a sabotage one; achieving 6.7%~20% and 50.7% ~ 138% higher unicast throughput and broadcast delivery success rate over regular benign agents, respectively.