Q-learning with function Approximator for clustering based Optimal resource Allocation in fog environment

Abstract

Fog computing is a new paradigm for delivering services close to the user. The exponential growth of IoT devices and big data complicates fog resource distribution. Inefficient resource allocation can result in resource scarcity and the inability to finish a task assignment on time. As a result, correct allocation is required to improve the efficiency of fog resources. Resource allocation is a difficult task with heterogeneous constraint resources. As fog computing deals with real-time data, therefore, needs resource allocation in real-time that increases the necessity of having appropriate and optimal resource allocation in real-time. Therefore, this paper targets optimal resource allocation. To address the resource allocation problem, Q-learning with function Approximator for clustering based Optimal resource Allocation (QL(FA)-CORA) model is designed, considering the problem as a decision making problem, reinforcement learning method is used to solve it. Problem formulation is done using the Markov decision process. Clustering is done to reduce the service time. Proposed an optimal resource allocation using the QL function approximator (ORA- QLFA) algorithm. to enhance the efficiency and performance of the proposed fog environment. Simulations are done to evaluate the validation of the proposed algorithm. Also, comparisons are made with linear Q networks using different parameters such as expected discounted return, maximum steps taken by the fog resource controller, etc. Simulation results show the proposed algorithm performs better in all the cases and converged to optimal results after a few iterations rather than a linear Q network.