{"title":"Improved Exploration Strategy for Q-Learning Based Multipath Routing in SDN Networks","authors":"","doi":"10.1007/s10922-024-09804-0","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Software-Defined Networking (SDN) is characterized by a high level of programmability and offers a rich set of capabilities for network management operations. Network intelligence is centralized in the controller, which is responsible for updating the routing policies according to the applications’ requirements. To further enhance such capabilities, the controller has to be endowed with intelligence by integrating Artificial Intelligence (AI) tools in order to provide the controller the ability to autonomously reconfigure the network in a timely way. In this paper, we address the deployment of a Q-learning algorithm for the routing optimization problem in terms of latency minimization. Using a direct modeling approach of the multi-path flow-routing problem, we delve deeper into the impact of the exploration-exploitation strategies on the algorithm’s performance. Furthermore, we propose a couple of improvements to the Q-Learning algorithm to enhance its performance within the considered environment. On the one hand, we integrate a congestion-avoidance mechanism in the exploration phase, which leads to effective improvements in the algorithm’s performance with regard to average latency, convergence time, and computation time. On the other hand, we propose to implement a novel strategy based on the Max-Boltzman Exploration method (MBE), which is a combination of the traditional <span> <span>\\(\\varepsilon\\)</span> </span>- greedy and softmax strategies. The results show that, for an appropriate tuning of the hyperparameters, the MBE strategy combined with the congestion-avoidance mechanism performs better than the <span> <span>\\(\\varepsilon\\)</span> </span>-greedy, <span> <span>\\(\\varepsilon\\)</span> </span>-decay, and Softmax strategies in terms of average latency, convergence time, and computation time.</p>","PeriodicalId":50119,"journal":{"name":"Journal of Network and Systems Management","volume":"18 1","pages":""},"PeriodicalIF":4.1000,"publicationDate":"2024-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Network and Systems Management","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s10922-024-09804-0","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Software-Defined Networking (SDN) is characterized by a high level of programmability and offers a rich set of capabilities for network management operations. Network intelligence is centralized in the controller, which is responsible for updating the routing policies according to the applications’ requirements. To further enhance such capabilities, the controller has to be endowed with intelligence by integrating Artificial Intelligence (AI) tools in order to provide the controller the ability to autonomously reconfigure the network in a timely way. In this paper, we address the deployment of a Q-learning algorithm for the routing optimization problem in terms of latency minimization. Using a direct modeling approach of the multi-path flow-routing problem, we delve deeper into the impact of the exploration-exploitation strategies on the algorithm’s performance. Furthermore, we propose a couple of improvements to the Q-Learning algorithm to enhance its performance within the considered environment. On the one hand, we integrate a congestion-avoidance mechanism in the exploration phase, which leads to effective improvements in the algorithm’s performance with regard to average latency, convergence time, and computation time. On the other hand, we propose to implement a novel strategy based on the Max-Boltzman Exploration method (MBE), which is a combination of the traditional \(\varepsilon\)- greedy and softmax strategies. The results show that, for an appropriate tuning of the hyperparameters, the MBE strategy combined with the congestion-avoidance mechanism performs better than the \(\varepsilon\)-greedy, \(\varepsilon\)-decay, and Softmax strategies in terms of average latency, convergence time, and computation time.
期刊介绍：
Journal of Network and Systems Management, features peer-reviewed original research, as well as case studies in the fields of network and system management. The journal regularly disseminates significant new information on both the telecommunications and computing aspects of these fields, as well as their evolution and emerging integration. This outstanding quarterly covers architecture, analysis, design, software, standards, and migration issues related to the operation, management, and control of distributed systems and communication networks for voice, data, video, and networked computing.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
