Reduced state space and cost function in reinforcement learning for demand response control of multiple EV charging stations

Manu Lahariya, Nasrin Sadeghianpourhamami, Chris Develder
{"title":"Reduced state space and cost function in reinforcement learning for demand response control of multiple EV charging stations","authors":"Manu Lahariya, Nasrin Sadeghianpourhamami, Chris Develder","doi":"10.1145/3360322.3360992","DOIUrl":null,"url":null,"abstract":"Electric vehicle (EV) charging stations represent a substantial load with significant flexibility. Balancing such load with model-free demand response (DR) based on reinforcement learning (RL) is an attractive approach. We build on previous RL research using a Markov decision process (MDP) to simultaneously coordinate multiple charging stations. The previously proposed approach is computationally expensive in terms of large training times, limiting its feasibility and practicality. We propose to a priori force the control policy to always fulfill any charging demand that does not offer any flexibility at a given point, and thus use an updated cost function. We compare the policy of the newly proposed approach with the original (costly) one, for the case of load flattening, in terms of (i) processing time to learn the RL-based charging policy, and (ii) overall performance of the policy decisions in terms of meeting the target load for unseen test data.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"14 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"7","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3360322.3360992","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 7

Abstract

Electric vehicle (EV) charging stations represent a substantial load with significant flexibility. Balancing such load with model-free demand response (DR) based on reinforcement learning (RL) is an attractive approach. We build on previous RL research using a Markov decision process (MDP) to simultaneously coordinate multiple charging stations. The previously proposed approach is computationally expensive in terms of large training times, limiting its feasibility and practicality. We propose to a priori force the control policy to always fulfill any charging demand that does not offer any flexibility at a given point, and thus use an updated cost function. We compare the policy of the newly proposed approach with the original (costly) one, for the case of load flattening, in terms of (i) processing time to learn the RL-based charging policy, and (ii) overall performance of the policy decisions in terms of meeting the target load for unseen test data.
基于强化学习的多充电站需求响应控制的简化状态空间和成本函数
电动汽车(EV)充电站代表了大量的负载,具有很大的灵活性。利用基于强化学习(RL)的无模型需求响应(DR)来平衡这种负载是一种很有吸引力的方法。我们在之前的RL研究的基础上,使用马尔可夫决策过程(MDP)来同时协调多个充电站。先前提出的方法由于训练时间大,计算成本高,限制了其可行性和实用性。我们建议先验地强制控制策略总是满足在给定点不提供任何灵活性的任何充电需求,从而使用更新的成本函数。在负载平坦的情况下,我们将新提出的策略与原始的(昂贵的)策略进行比较,包括(i)学习基于rl的收费策略的处理时间,以及(ii)在满足未知测试数据的目标负载方面策略决策的总体性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信