Intelligent demand response for electricity consumers: A multi-armed bandit game approach

Real-time electricity pricing (RTP) for consumers has long been argued to be key to realize the many envisioned benefits of a smart energy grid. How to actually implement an RTP scheme, however, is still under debate. Since most of the organized wholesale power markets in the US implement a two-settlement system, with day-ahead electricity price forecasts guiding financial and physical transactions in the next day and real-time ex post prices settling any real-time imbalances, it is a natural idea to let consumers respond to the day-ahead prices. Such an idea, however, may lead to consumers all respond in the same fashion, causing large swings of the energy demand and prices, which may jeopardize system stability and increase consumers' financial risks. To overcome this issue, we propose a game-theoretic framework in which each consumer solves a multi-armed bandit problem; that is, consumers learn from the history and attempts to minimize their regrets. The consequence is drastically reduced volatility on real-time prices and much flatter load curves for the entire grid. Such results are not only based on simulation, but are also supported by theories of mean-field equilibria in multi-armed bandit games.