A multi-armed bandit formulation for distributed appliances scheduling in smart grids

Abstract

Game-theoretic Demand-Side Management (DSM) systems represent a promising solution to control the electrical appliances of residential consumers. Such frameworks allow indeed for the optimal management of loads without any centralized coordination since decisions are taken locally and directly by users. In this paper, we focus our analysis on a game-theoretic DSM framework designed to reduce the bill of a group of users. In order to converge to the equilibrium of the game, we adopt an efficient learning algorithm proposed in the literature, Exp3, along with two variants that we propose to speed up convergence. In defining these methods, we model the appliances scheduling problem as a Multi-Armed Bandit (MAB) problem, a classical formulation of decision theory. We analyze the proposed learning methods based on realistic instances in several use-case scenarios and show numerically their effectiveness in improving the performance of next generation smart grid systems.