{"title":"基于深度强化学习算法中改进的近端策略优化的 FCHEV 自适应能源管理策略","authors":"","doi":"10.1016/j.enconman.2024.118977","DOIUrl":null,"url":null,"abstract":"<div><p>In order to reduce hydrogen consumption, relieve degradation of fuel cells, and improve the operational efficiency of fuel cell hybrid electric vehicles, an energy management strategy is proposed. This strategy is based on an improved proximal policy optimization (PPO) deep reinforcement learning algorithm. A hierarchical optimal control strategy with adaptive driving pattern is constructed by using clipping strategy instead of divergence penalty. This strategy can reduce the number of iterations of the algorithm. To improve the accuracy of driving pattern identification, in the identification layer, a multilayer perceptron neural network model is used for offline training of driving pattern recognition. Online driving pattern recognition is carried out using a sliding recognition window method. At the policy layer, in order to improve system response speed and reduce the impact of peak power on the fuel cell, dynamic programming is used for offline optimization of lithium battery state of charge values under different driving patterns. Then, based on the optimized objective of equivalent hydrogen consumption normalized by actual hydrogen consumption and fuel cell degradation, the lithium battery is charged and discharged online according to the corresponding optimal state-of-charge (SoC) value for each driving pattern. The experimental results show that the accuracy of driving pattern recognition can reach 90.75%, and the economic performance has improved by 3.11%. Therefore, this study can effectively balance hydrogen consumption and fuel cell degradation to achieve hydrogen conservation and delay in fuel cell lifespan degradation.</p></div>","PeriodicalId":11664,"journal":{"name":"Energy Conversion and Management","volume":null,"pages":null},"PeriodicalIF":9.9000,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Adaptive energy management strategy for FCHEV based on improved proximal policy optimization in deep reinforcement learning algorithm\",\"authors\":\"\",\"doi\":\"10.1016/j.enconman.2024.118977\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>In order to reduce hydrogen consumption, relieve degradation of fuel cells, and improve the operational efficiency of fuel cell hybrid electric vehicles, an energy management strategy is proposed. This strategy is based on an improved proximal policy optimization (PPO) deep reinforcement learning algorithm. A hierarchical optimal control strategy with adaptive driving pattern is constructed by using clipping strategy instead of divergence penalty. This strategy can reduce the number of iterations of the algorithm. To improve the accuracy of driving pattern identification, in the identification layer, a multilayer perceptron neural network model is used for offline training of driving pattern recognition. Online driving pattern recognition is carried out using a sliding recognition window method. At the policy layer, in order to improve system response speed and reduce the impact of peak power on the fuel cell, dynamic programming is used for offline optimization of lithium battery state of charge values under different driving patterns. Then, based on the optimized objective of equivalent hydrogen consumption normalized by actual hydrogen consumption and fuel cell degradation, the lithium battery is charged and discharged online according to the corresponding optimal state-of-charge (SoC) value for each driving pattern. The experimental results show that the accuracy of driving pattern recognition can reach 90.75%, and the economic performance has improved by 3.11%. Therefore, this study can effectively balance hydrogen consumption and fuel cell degradation to achieve hydrogen conservation and delay in fuel cell lifespan degradation.</p></div>\",\"PeriodicalId\":11664,\"journal\":{\"name\":\"Energy Conversion and Management\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":9.9000,\"publicationDate\":\"2024-09-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Energy Conversion and Management\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S019689042400918X\",\"RegionNum\":1,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENERGY & FUELS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Energy Conversion and Management","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S019689042400918X","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENERGY & FUELS","Score":null,"Total":0}
Adaptive energy management strategy for FCHEV based on improved proximal policy optimization in deep reinforcement learning algorithm
In order to reduce hydrogen consumption, relieve degradation of fuel cells, and improve the operational efficiency of fuel cell hybrid electric vehicles, an energy management strategy is proposed. This strategy is based on an improved proximal policy optimization (PPO) deep reinforcement learning algorithm. A hierarchical optimal control strategy with adaptive driving pattern is constructed by using clipping strategy instead of divergence penalty. This strategy can reduce the number of iterations of the algorithm. To improve the accuracy of driving pattern identification, in the identification layer, a multilayer perceptron neural network model is used for offline training of driving pattern recognition. Online driving pattern recognition is carried out using a sliding recognition window method. At the policy layer, in order to improve system response speed and reduce the impact of peak power on the fuel cell, dynamic programming is used for offline optimization of lithium battery state of charge values under different driving patterns. Then, based on the optimized objective of equivalent hydrogen consumption normalized by actual hydrogen consumption and fuel cell degradation, the lithium battery is charged and discharged online according to the corresponding optimal state-of-charge (SoC) value for each driving pattern. The experimental results show that the accuracy of driving pattern recognition can reach 90.75%, and the economic performance has improved by 3.11%. Therefore, this study can effectively balance hydrogen consumption and fuel cell degradation to achieve hydrogen conservation and delay in fuel cell lifespan degradation.
期刊介绍:
The journal Energy Conversion and Management provides a forum for publishing original contributions and comprehensive technical review articles of interdisciplinary and original research on all important energy topics.
The topics considered include energy generation, utilization, conversion, storage, transmission, conservation, management and sustainability. These topics typically involve various types of energy such as mechanical, thermal, nuclear, chemical, electromagnetic, magnetic and electric. These energy types cover all known energy resources, including renewable resources (e.g., solar, bio, hydro, wind, geothermal and ocean energy), fossil fuels and nuclear resources.