A Contextual Multi-Armed Bandit approach for NDN forwarding

Abstract

Named Data Networking (NDN) is a promising Internet architecture that aims to supersede the current IP-based architecture and shift the host-centric model to a data-centric one. Within NDN, forwarding Interest packets remains a significant challenge and has attracted considerable recent research attention. The momentum behind machine learning techniques, especially reinforcement learning, is steadily growing, offering the potential to deliver intelligent, adaptable, and reliable NDN forwarding algorithms. In this context, this paper proposes efficient NDN forwarding strategies based on Contextual Multi-Armed Bandit (CMAB). Initially, we employ CMAB to address the challenge of forwarding Interest packets and introduce a new CMAB model tailored for NDN, dubbed CMAB4NDN. Subsequently, we construct the CMAB context using information derived from the content name and the network congestion state, which are then fed into the CMAB4NDN learning algorithm to decide on the best forwarding action. Further, we develop three CMAB strategies, namely Lin-$\varepsilon$-Greedy, Linear Upper Confidence Bound, and Contextual Thompson Sampling, and deploy them within our proposal. CMAB4NDN was implemented in ndnSIM, thoroughly evaluated, and compared with multiple state-of-the-art NDN forwarding algorithms across various scenarios. The obtained results confirm the relevance and superiority of our approach in terms of delay, throughput, and packet loss.