{"title":"Reinforcement Learning Based Bidding Method with High-dimensional Bids in Electricity Markets","authors":"Jinyu Liu;Hongye Guo;Yun Li;Qinghu Tang;Fuquan Huang;Tunan Chen;Haiwang Zhong","doi":"10.35833/MPCE.2024.000811","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000811","url":null,"abstract":"Over the past decade, bidding in electricity markets has attracted widespread attention. Reinforcement learning (RL) has been widely used for electricity market bidding as a powerful artificial intelligence (AI) tool to make decisions under real-world uncertainties. However, current RL-based bidding methods mostly employ low-dimensional bids (LDBs), which significantly diverge from the <tex>$N$</tex> price-power pairs commonly used in current electricity markets. The <tex>$N$</tex>-pair bid format is denoted as high-dimensional bid (HDB) format, which has not been fully integrated into the existing RL-based bidding methods. The loss of flexibility of current RL-based bidding methods could greatly limit the bidding profits and make it difficult to address the increasing uncertainties caused by renewable energy generation. In this paper, we propose a framework for fully utilizing HDBs in RL-based bidding methods. First, we employ a special type of neural network called the neural network supply function (NNSF) to generate HDBs in the form of <tex>$N$</tex> price-power pairs. Second, we embed the NNSF into a Markov decision process (MDP) to make it compatible with most existing RL algorithms. Finally, the experiments on energy storage systems (ES-Ss) in the Pennsylvania-New Jersey-Maryland (PJM) real-time electricity market show that the proposed bidding method with HDBs can increase the bidding flexibility, thereby increasing the profits of state-of-the-art RL-based bidding methods.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 4","pages":"1373-1382"},"PeriodicalIF":5.7,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10856824","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144716308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ze Hu;Peijun Zheng;Ka Wing Chan;Siqi Bu;Ziqing Zhu;Xiang Wei;Yosuke Nakanishi
{"title":"A Hybrid Data-Driven Approach Integrating Temporal Fusion Transformer and Soft Actor-Critic Algorithm for Optimal Scheduling of Building Integrated Energy Systems","authors":"Ze Hu;Peijun Zheng;Ka Wing Chan;Siqi Bu;Ziqing Zhu;Xiang Wei;Yosuke Nakanishi","doi":"10.35833/MPCE.2024.000909","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000909","url":null,"abstract":"Building integrated energy systems (BIESs) are pivotal for enhancing energy efficiency by accounting for a significant proportion of global energy consumption. Two key barriers that reduce the BIES operational efficiency mainly lie in the renewable generation uncertainty and operational non-convexity of combined heat and power (CHP) units. To this end, this paper proposes a soft actor-critic (SAC) algorithm to solve the scheduling problem of BIES, which overcomes the model non-convexity and shows advantages in robustness and generalization. This paper also adopts a temporal fusion transformer (TFT) to enhance the optimal solution for the SAC algorithm by forecasting the renewable generation and energy demand. The TFT can effectively capture the complex temporal patterns and dependencies that span multiple steps. Furthermore, its forecasting results are interpretable due to the employment of a self-attention layer so as to assist in more trustworthy decision-making in the SAC algorithm. The proposed hybrid data-driven approach integrating TFT and SAC algorithm, i.e., TFT-SAC approach, is trained and tested on a real-world dataset to validate its superior performance in reducing the energy cost and computational time compared with the benchmark approaches. The generalization performance for the scheduling policy, as well as the sensitivity analysis, are examined in the case studies.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 3","pages":"878-891"},"PeriodicalIF":5.7,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10856822","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144139844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Interval Demand Response Potential Evaluation and Risk Dispatch to Incorporate Public Buildings into Power System Operation","authors":"Yu Yao;Chengjin Ye;Yuming Zhao;Yi Ding","doi":"10.35833/MPCE.2024.000919","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000919","url":null,"abstract":"Public buildings present substantial demand response (DR) potential, which can participate in the power system operation. However, most public buildings exhibit a high degree of uncertainties due to incomplete information, varying thermal parameters, and stochastic user behaviors, which hinders incorporating the public buildings into power system operation. To address the problem, this paper proposes an interval DR potential evaluation method and a risk dispatch model to integrate public buildings with uncertainties into power system operation. Firstly, the DR evaluation is developed based on the equivalent thermal parameter (ETP) model, actual outdoor temperature data, and air conditioning (AC) consumption data. To quantify the uncertainties of public buildings, the interval evaluation is given employing the linear regression method considering the confidence bound. Utilizing the evaluation results, the risk dispatch model is proposed to allocate public building reserve based on the chance constrained programming (CCP). Finally, the proposed risk dispatch model is reformulated to a mixed-integer second-order cone programming (MISOCP) for its solution. The proposed evaluation method and the risk dispatch model are validated based on the modified IEEE 39-bus system and actual building data obtained from a southern city in China.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 4","pages":"1347-1359"},"PeriodicalIF":5.7,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10856823","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144716307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Frequency Deadband Control of Grid-forming Energy Storage Inverter in Primary Frequency Regulation","authors":"Wei Zhang;Zhenxiong Wang;Yingjie Peng;Jingting Wu;Qiru Li;Hao Yi;Zebin Yang;Li Li;Fang Zhuo","doi":"10.35833/MPCE.2024.000757","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000757","url":null,"abstract":"With the increased penetration of renewable energy sources, the grid-forming (GFM) energy storage (ES) has been considered to engage in primary frequency regulation (PFR), often necessitating the use of a frequency deadband (FDB) to prevent excessive battery charging cycling and miti-gate frequency oscillations. Implementing the FDB is relatively straightforward in grid-following (GFL) control. However, implementing the FDB in GFM control presents a significant challenge since the inverter must abstain from providing active power at any frequency within the FDB. Therefore, in this paper, the performance of PFR control in the GFM-ES inverter is analyzed in detail first. Then, the FDB is implemented for GFM inverters with various types of synchronization methods, and the need for inertia response is also considered. Moreover, given the risk of oscillations near the FDB boundary, different FDB setting methods are proposed and examined, where an improved triangular hysteresis method is proposed to realize the fast response and enhanced stability. Finally, the simulation and experiment results are provided to verify the effectiveness of the above methods.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 1","pages":"167-178"},"PeriodicalIF":5.7,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10855741","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Guest Editorial: Special Section on Dynamic Performance and Flexibility Enhancement of RES-dominated Power Systems with Grid-forming Converters","authors":"Hassan Bevrani;Federico Milano","doi":"10.35833/MPCE.2025.000013","DOIUrl":"https://doi.org/10.35833/MPCE.2025.000013","url":null,"abstract":"Inverter-based distributed generation (DG) and renewable energy sources (RESs) are recognized as key elements to address the challenge of economically harvesting energy while, at the same time, achieving net-zero targets. However, inverter-based generation decreases the physical inertia available in the system and increases uncertainty. Moreover, recent studies have highlighted that relatively high integration of DGs/RESs can negatively impact on power grid dynamics, power quality, frequency control, voltage regulation, as well as other control and operational issues. All these issues significantly limit the penetration of DG and RESs.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 1","pages":"1-2"},"PeriodicalIF":5.7,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10855698","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"DC Voltage Control with Grid-Forming Capability for Enhancing Stability of HVDC System","authors":"Ghazala Shafique;Johan Boukhenfouf;François Gruson;Frédéric Colas;Xavier Guillaud","doi":"10.35833/MPCE.2024.000822","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000822","url":null,"abstract":"Grid-forming (GFM) converters are recognized for their stabilizing effects in renewable energy systems. Integrating GFM converters into high-voltage direct current (HVDC) systems requires DC voltage control. However, there can be a conflict between GFM converter and DC voltage control when they are used in combination. This paper presents a rigorous control design for a GFM converter that connects the DC-link voltage to the power angle of the converter, thereby integrating DC voltage control with GFM capability. The proposed control is validated through small-signal and transient-stability analyses on a modular multilevel converter (MMC)-based HVDC system with a point-to-point (P2P) GFM-GFM configuration. The results demonstrate that employing a GFM-GFM configuration with the proposed control enhances the stability of the AC system to which it is connected. The system exhibits low sensitivity to grid strength and can sustain islanding conditions. The high stability limit of the system with varying grid strength using the proposed control is validated using a system with four voltage source converters.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 1","pages":"66-78"},"PeriodicalIF":5.7,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10855700","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaokuan Jin;Jianhua Wang;Han Yan;Xijun Ni;Zhendong Ji;Baojian Ji;Ding Wan
{"title":"Series-Parallel Sequence Impedance Models of Multi-Loop Grid-Forming Converters","authors":"Xiaokuan Jin;Jianhua Wang;Han Yan;Xijun Ni;Zhendong Ji;Baojian Ji;Ding Wan","doi":"10.35833/MPCE.2024.000676","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000676","url":null,"abstract":"The gradual penetration of grid-forming (GFM) converters into new power systems with renewable energy sources may result in the emergence of small-signal instability issues. These issues can be elucidated using sequence impedance models, which offer a more tangible and meaningful interpretation than dq-domain impedance models and state-space models. However, existing research has primarily focused on the impact of power loops and inner control loops in GFM converters, which has not yet elucidated the precise physical interpretation of inner voltage and current loops of GFM converters in circuits. This paper derives series-parallel sequence impedance models of multi-loop GFM converters, demonstrating that the voltage loop can be regarded as a parallel impedance and the current loop as a series impedance. Consequently, the corre-sponding small-signal stability characteristics can be identified through Bode diagrams of sequence impedances or by examining the physical meanings of impedances in series and in parallel. The results indicate that the GFM converter with a single power loop is a candidate suitable for application in new power systems, given its reduced number of control parameters and enhanced low-frequency performance, particularly in weak grids. The results of PLECS simulations and corresponding prototype experiments verify the accuracy of the analytical analysis under diverse grid conditions.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 1","pages":"29-41"},"PeriodicalIF":5.7,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10855724","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dynamic Analysis of Uniformity and Difference for Grid-following and Grid-forming Voltage Source Converters Using Phasor and Topological Homology Methods","authors":"Haiyu Zhao;Hongyu Zhou;Wei Yao;Qihang Zong;Jinyu Wen","doi":"10.35833/MPCE.2024.000722","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000722","url":null,"abstract":"Grid-following voltage source converter (GFL-VSC) and grid-forming voltage source converter (GFM-VSC) have different dynamic characteristics for active power-frequency and reactive power-voltage supports of the power grid. This paper aims to clarify and recognize the difference between grid-following (GFL) and grid-forming (GFM) frequency-voltage support more intuitively and clearly. Firstly, the phasor model considering circuit constraints is established based on the port circuit equations of the converter. It is revealed that the voltage and active power linearly correspond to the horizontal and vertical axes in the phasor space referenced to the grid voltage pha-sore Secondly, based on topological homology, GFL and GFM controls are transformed and mapped into different trajectories. The topological similarity of the characteristic curves for GFL and GFM controls is the essential cause of their uniformity. Based on the above model, it is indicated that GFL-VSC and GFM-VSC possess uniformity with regard to active power response, type of coupling, and phasor trajectory. They differ in synchronization, power coupling mechanisms, dynamics, and active power-voltage operation domain in the quasi-steady state. Case studies are undertaken on GFL-VSC and GFM-VSC integrated into a four-machine two-area system. Simulation results verify that the dynamic uniformity and difference of GFL-VSC and GFM-VSC are intuitively and comprehensively revealed.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 1","pages":"3-14"},"PeriodicalIF":5.7,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10855723","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143183953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Francisco Jesús Matas-Díaz;Manuel Barragán-Villarejo;José María Maza-Ortega
{"title":"A Systematic Small-signal Analysis Procedure for Improving Synchronization Stability of Grid-forming Virtual Synchronous Generators","authors":"Francisco Jesús Matas-Díaz;Manuel Barragán-Villarejo;José María Maza-Ortega","doi":"10.35833/MPCE.2024.000316","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000316","url":null,"abstract":"The integration of converter-interfaced generators (CIGs) into power systems is rapidly replacing traditional synchronous machines. To ensure the security of power supply, modern power systems require the application of grid-forming technologies. This study presents a systematic small-signal analysis procedure to assess the synchronization stability of gridforming virtual synchronous generators (VSGs) considering the power system characteristics. Specifically, this procedure offers guidance in tuning controller gains to enhance stability. It is applied to six different grid-forming VSGs and experimentally tested to validate the theoretical analysis. This study concludes with key findings and a discussion on the suitability of the analyzed grid-forming VSGs based on the power system characteristics.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 1","pages":"102-114"},"PeriodicalIF":5.7,"publicationDate":"2025-01-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10855740","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143184056","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qing Sun;Junjie Tang;Sui Peng;Weijie Zhong;Liu Zhu;Yuan Zhao;Wenyuan Li
{"title":"Operational Reliability Evaluation and Risk Mitigation of Asynchronous Grids Coupled Through Flexible HVDC Systems","authors":"Qing Sun;Junjie Tang;Sui Peng;Weijie Zhong;Liu Zhu;Yuan Zhao;Wenyuan Li","doi":"10.35833/MPCE.2024.000677","DOIUrl":"https://doi.org/10.35833/MPCE.2024.000677","url":null,"abstract":"This paper constructs a synthetic framework for the operational reliability evaluation and risk mitigation of asynchronous grids coupled through flexible high-voltage DC (HVDC) systems (AGs-FDCSs). First of all, an analytical model for the unavailability of DC units is reformulated to refine and facilitate the reliability modeling of such flexible HVDC systems considering their time-dependent features as well as the impacts of converter station configurations. Subsequently, the operational risk associated with the redispatch procedure is extended to the reliability evaluation of composite power system, and the risk is mitigated through an optimal power flow (OPF) based short-term state assessment model. In addition, some new reliability indices like expected DC transmission power (EDCTP) and DC terminal outage probability (DCTOP) are defined to quantify the impact of the reliability of flexible HVDC systems on the entire grid. The effectiveness of the proposed framework on a modified IEEE RTS-79 system is validated with the elabo-rate discussions on the time-dependent reliability of AGs-FDC-Ss as well as the impacts of the converter station configurations.","PeriodicalId":51326,"journal":{"name":"Journal of Modern Power Systems and Clean Energy","volume":"13 5","pages":"1701-1713"},"PeriodicalIF":6.1,"publicationDate":"2025-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10851867","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145089997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}