Double Learning for Suppliers' Bidding Strategy in the Electricity Market

In this article, we have simulated an agent-based model to investigate the bidding behavior of generation companies (GenCos) in the electricity market under different pricing rules. In a real electricity market, they do not have complete information about the behavior of competitors and therefore make their own decisions based on information that exists on the market-clearing price (MCP) from the past. So considering of high ability of reinforcement learning (RL) algorithms for making decisions on issues with incomplete information, the behavior of companies has been simulated with RL algorithms. In addition, to remove the maximization bias in usual algorithms, we have used the double learning technique for unbiased estimation. The results have shown that companies using Double Q-learning and Double SARSA algorithms, can gain optimal values with unbiased estimation. Furthermore, we investigated market-clearing mechanisms with different pricing rules under an equal rationing policy, in terms of competitiveness in the market and total profit of GenCos. The results showed that in terms of competitiveness, the uniform pricing rule causes more competitive bid prices. However, from the side of the total profit of GenCos, it was seen that with changing from the uniform pricing to the pay-as-bid (P AB), the total profit of companies would reduce.