{"title":"评估多个 DER 聚合器对能源批发市场的影响:混合均值场方法","authors":"Jun He, Andrew L. Liu","doi":"arxiv-2409.00107","DOIUrl":null,"url":null,"abstract":"The integration of distributed energy resources (DERs) into wholesale energy\nmarkets can greatly enhance grid flexibility, improve market efficiency, and\ncontribute to a more sustainable energy future. As DERs -- such as solar PV\npanels and energy storage -- proliferate, effective mechanisms are needed to\nensure that small prosumers can participate meaningfully in these markets. We\nstudy a wholesale market model featuring multiple DER aggregators, each\ncontrolling a portfolio of DER resources and bidding into the market on behalf\nof the DER asset owners. The key of our approach lies in recognizing the\nrepeated nature of market interactions the ability of participants to learn and\nadapt over time. Specifically, Aggregators repeatedly interact with each other\nand with other suppliers in the wholesale market, collectively shaping\nwholesale electricity prices (aka the locational marginal prices (LMPs)). We\nmodel this multi-agent interaction using a mean-field game (MFG), which uses\nmarket information -- reflecting the average behavior of market participants --\nto enable each aggregator to predict long-term LMP trends and make informed\ndecisions. For each aggregator, because they control the DERs within their\nportfolio under certain contract structures, we employ a mean-field control\n(MFC) approach (as opposed to a MFG) to learn an optimal policy that maximizes\nthe total rewards of the DERs under their management. We also propose a\nreinforcement learning (RL)-based method to help each agent learn optimal\nstrategies within the MFG framework, enhancing their ability to adapt to market\nconditions and uncertainties. Numerical simulations show that LMPs quickly\nreach a steady state in the hybrid mean-field approach. Furthermore, our\nresults demonstrate that the combination of energy storage and mean-field\nlearning significantly reduces price volatility compared to scenarios without\nstorage.","PeriodicalId":501273,"journal":{"name":"arXiv - ECON - General Economics","volume":"24 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Evaluating the Impact of Multiple DER Aggregators on Wholesale Energy Markets: A Hybrid Mean Field Approach\",\"authors\":\"Jun He, Andrew L. Liu\",\"doi\":\"arxiv-2409.00107\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The integration of distributed energy resources (DERs) into wholesale energy\\nmarkets can greatly enhance grid flexibility, improve market efficiency, and\\ncontribute to a more sustainable energy future. As DERs -- such as solar PV\\npanels and energy storage -- proliferate, effective mechanisms are needed to\\nensure that small prosumers can participate meaningfully in these markets. We\\nstudy a wholesale market model featuring multiple DER aggregators, each\\ncontrolling a portfolio of DER resources and bidding into the market on behalf\\nof the DER asset owners. The key of our approach lies in recognizing the\\nrepeated nature of market interactions the ability of participants to learn and\\nadapt over time. Specifically, Aggregators repeatedly interact with each other\\nand with other suppliers in the wholesale market, collectively shaping\\nwholesale electricity prices (aka the locational marginal prices (LMPs)). We\\nmodel this multi-agent interaction using a mean-field game (MFG), which uses\\nmarket information -- reflecting the average behavior of market participants --\\nto enable each aggregator to predict long-term LMP trends and make informed\\ndecisions. For each aggregator, because they control the DERs within their\\nportfolio under certain contract structures, we employ a mean-field control\\n(MFC) approach (as opposed to a MFG) to learn an optimal policy that maximizes\\nthe total rewards of the DERs under their management. We also propose a\\nreinforcement learning (RL)-based method to help each agent learn optimal\\nstrategies within the MFG framework, enhancing their ability to adapt to market\\nconditions and uncertainties. Numerical simulations show that LMPs quickly\\nreach a steady state in the hybrid mean-field approach. Furthermore, our\\nresults demonstrate that the combination of energy storage and mean-field\\nlearning significantly reduces price volatility compared to scenarios without\\nstorage.\",\"PeriodicalId\":501273,\"journal\":{\"name\":\"arXiv - ECON - General Economics\",\"volume\":\"24 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - ECON - General Economics\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.00107\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - ECON - General Economics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.00107","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
将分布式能源资源(DER)纳入能源批发市场,可以大大提高电网的灵活性,提高市场效率,并有助于实现更可持续的能源未来。随着太阳能光伏板和储能等 DER 的激增,需要建立有效的机制来确保小型消费者能够有意义地参与这些市场。Westudy 的批发市场模式以多个 DER 聚合器为特色,每个聚合器控制一个 DER 资源组合,并代表 DER 资产所有者参与市场竞标。我们的方法的关键在于认识到市场互动的反复性以及参与者随时间学习和适应的能力。具体来说,聚合器在批发市场中与其他供应商反复互动,共同影响批发电价(又称本地边际价格 (LMP))。我们使用均场博弈(MFG)来模拟这种多代理互动,该博弈使用市场信息(反映市场参与者的平均行为),使每个聚合者都能预测 LMP 的长期趋势,并做出明智的决策。对于每个聚合器而言,由于它们根据特定的合同结构控制其投资组合中的 DER,因此我们采用均场控制(MFC)方法(而非 MFG)来学习最优策略,使其管理下的 DER 的总回报最大化。我们还提出了基于强化学习(RL)的方法,以帮助每个代理在 MFG 框架内学习最优策略,从而增强其适应市场条件和不确定性的能力。数值模拟表明,在混合均值场方法中,LMPs 很快就能达到稳定状态。此外,我们的研究结果表明,与没有储能的情况相比,储能和均值场学习的结合大大降低了价格波动。
Evaluating the Impact of Multiple DER Aggregators on Wholesale Energy Markets: A Hybrid Mean Field Approach
The integration of distributed energy resources (DERs) into wholesale energy
markets can greatly enhance grid flexibility, improve market efficiency, and
contribute to a more sustainable energy future. As DERs -- such as solar PV
panels and energy storage -- proliferate, effective mechanisms are needed to
ensure that small prosumers can participate meaningfully in these markets. We
study a wholesale market model featuring multiple DER aggregators, each
controlling a portfolio of DER resources and bidding into the market on behalf
of the DER asset owners. The key of our approach lies in recognizing the
repeated nature of market interactions the ability of participants to learn and
adapt over time. Specifically, Aggregators repeatedly interact with each other
and with other suppliers in the wholesale market, collectively shaping
wholesale electricity prices (aka the locational marginal prices (LMPs)). We
model this multi-agent interaction using a mean-field game (MFG), which uses
market information -- reflecting the average behavior of market participants --
to enable each aggregator to predict long-term LMP trends and make informed
decisions. For each aggregator, because they control the DERs within their
portfolio under certain contract structures, we employ a mean-field control
(MFC) approach (as opposed to a MFG) to learn an optimal policy that maximizes
the total rewards of the DERs under their management. We also propose a
reinforcement learning (RL)-based method to help each agent learn optimal
strategies within the MFG framework, enhancing their ability to adapt to market
conditions and uncertainties. Numerical simulations show that LMPs quickly
reach a steady state in the hybrid mean-field approach. Furthermore, our
results demonstrate that the combination of energy storage and mean-field
learning significantly reduces price volatility compared to scenarios without
storage.