Monte-Carlo Tree Search Aided Contextual Online Learning Approach for Wireless Caching

Abstract

Caching popular contents at the edge of wireless networks has recently emerged as a promising technique to offload mobile data traffic and improve the quality of service for users. In the big-data era, the size of the content space is essentially infinite. Moreover, users with common features typically share similar content preference. In order to address these issues, we model the wireless caching problem as a contextual multi-armed bandit (CMAB) problem that considers the infinitely arms, and propose a Monte-Carlo tree search aided contextual upper confidence bound (MCTS-CUCB) algorithm, to make accurate content caching with low complexity. Specifically, we introduce a tree-based search method to analyze the content subspace instead of a single content, thereby reducing the computing load. In the search process, a cover tree is built in an incremental and asymmetric manner, which can reflect the users' content preference. Besides, contextualization allows to learn content preferences for groups of users having similar contexts, which significantly accelerates the learning process and improve the cache hit rate. Our simulation results on a real-world data set (MovieLens 1M Dataset) demonstrate that the proposed MCTS-CUCB algorithm is capable of achieving a considerable reduction in complexity compared with the existing related algorithms with a superior cache hit rate performance.