Deep recurrent Q-network algorithm for carbon emission allowance trading strategy

Abstract

Against the backdrop of global warming, the carbon trading market is considered as an effective means of emission reduction. With more and more companies and individuals participating in carbon markets for trading, it is of great theoretical and practical significance to help them automatically identify carbon trading investment opportunities and achieve intelligent carbon trading decisions. Based on the characteristics of the carbon trading market, we propose a novel deep reinforcement learning (DRL) trading strategy - Deep Recurrent Q-Network (DRQN). The experimental results show that the carbon allowance trading model based on the DRQN algorithm can provide optimal trading strategies and adapt to market changes. Specifically, the annualized returns for the DRQN algorithm strategy in the Guangdong (GD) and Hubei (HB) carbon markets are 15.43% and 34.75%, respectively, significantly outperforming other strategies. To better meet the needs of the actual implementation scenarios of the model, we analyze the impacts of discount factors and trading costs. The research results indicate that discount factors can provide participants with clearer expectations. In both carbon markets (GD and HB), there exists an optimal discount factor value of 0.4, as both excessively small or large values can have adverse effects on trading. Simultaneously, the government can ensure the fairness of carbon trading by regulating the costs of carbon trading to limit the speculative behavior of participants.