A hierarchical deep reinforcement learning method for coupled transportation and power distribution system dispatching

The randomness and dimensionality growth of variables in the Coupled transportation and power distribution systems (CTPS) pose challenges for effectively solving CTPS dispatching tasks. This paper presents a hierarchical deep reinforcement learning (HDRL) method, which disperses the action and state space of CTPS onto decision-making layer and autonomous optimization layer. The Cloud DRL model in the decision-making layer is responsible for the load assignment task of charging stations. The distribution network (DN) and transportation network (TN) DRL models in the autonomous optimization layer are responsible for optimizing the DN and TN respectively. A layer-wise training method is adopted to alleviate the asynchronous convergence problem of HDRL. Firstly, the Gurobi assists in achieving the efficient training of Cloud DRL model by ensuring the reward effectiveness of autonomous optimization layers. Meanwhile, the differential evolution (DE) algorithm assists in optimizing the diversity and focalization of the Transitions by controlling distribution patterns of species initialization, during the pre-sampling and training stage. Then, the trained Cloud DRL model is frozen to train the DN and TN DRL models. This method is tested on two different sizes of CTPS. Simulation analysis shows that this method improves the training performance of the HDRL model.