利用摊销式 Q 学习平衡充电状态的可重构电池控制算法

IF 4.6 4区 化学 Q2 ELECTROCHEMISTRY
Dominic Karnehm, Wolfgang Bliemetsrieder, Sebastian Pohlmann, Antje Neve
{"title":"利用摊销式 Q 学习平衡充电状态的可重构电池控制算法","authors":"Dominic Karnehm, Wolfgang Bliemetsrieder, Sebastian Pohlmann, Antje Neve","doi":"10.3390/batteries10040131","DOIUrl":null,"url":null,"abstract":"In the context of the electrification of the mobility sector, smart algorithms have to be developed to control battery packs. Smart and reconfigurable batteries are a promising alternative to conventional battery packs and offer new possibilities for operation and condition monitoring. This work proposes a reinforcement learning (RL) algorithm to balance the State of Charge (SoC) of reconfigurable batteries based on the topologies half-bridge and battery modular multilevel management (BM3). As an RL algorithm, Amortized Q-learning (AQL) is implemented, which enables the control of enormous numbers of possible configurations of the reconfigurable battery as well as the combination of classical controlling approaches and machine learning methods. This enhances the safety mechanisms during control. As a neural network of the AQL, a Feedforward Neuronal Network (FNN) is implemented consisting of three hidden layers. The experimental evaluation using a 12-cell hybrid cascaded multilevel converter illustrates the applicability of the method to balance the SoC and maintain the balanced state during discharge. The evaluation shows a 20.3% slower balancing process compared to a conventional approach. Nevertheless, AQL shows great potential for multiobjective optimizations and can be applied as an RL algorithm for control in power electronics.","PeriodicalId":8755,"journal":{"name":"Batteries","volume":null,"pages":null},"PeriodicalIF":4.6000,"publicationDate":"2024-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Controlling Algorithm of Reconfigurable Battery for State of Charge Balancing Using Amortized Q-Learning\",\"authors\":\"Dominic Karnehm, Wolfgang Bliemetsrieder, Sebastian Pohlmann, Antje Neve\",\"doi\":\"10.3390/batteries10040131\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In the context of the electrification of the mobility sector, smart algorithms have to be developed to control battery packs. Smart and reconfigurable batteries are a promising alternative to conventional battery packs and offer new possibilities for operation and condition monitoring. This work proposes a reinforcement learning (RL) algorithm to balance the State of Charge (SoC) of reconfigurable batteries based on the topologies half-bridge and battery modular multilevel management (BM3). As an RL algorithm, Amortized Q-learning (AQL) is implemented, which enables the control of enormous numbers of possible configurations of the reconfigurable battery as well as the combination of classical controlling approaches and machine learning methods. This enhances the safety mechanisms during control. As a neural network of the AQL, a Feedforward Neuronal Network (FNN) is implemented consisting of three hidden layers. The experimental evaluation using a 12-cell hybrid cascaded multilevel converter illustrates the applicability of the method to balance the SoC and maintain the balanced state during discharge. The evaluation shows a 20.3% slower balancing process compared to a conventional approach. Nevertheless, AQL shows great potential for multiobjective optimizations and can be applied as an RL algorithm for control in power electronics.\",\"PeriodicalId\":8755,\"journal\":{\"name\":\"Batteries\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":4.6000,\"publicationDate\":\"2024-04-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Batteries\",\"FirstCategoryId\":\"92\",\"ListUrlMain\":\"https://doi.org/10.3390/batteries10040131\",\"RegionNum\":4,\"RegionCategory\":\"化学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ELECTROCHEMISTRY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Batteries","FirstCategoryId":"92","ListUrlMain":"https://doi.org/10.3390/batteries10040131","RegionNum":4,"RegionCategory":"化学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ELECTROCHEMISTRY","Score":null,"Total":0}
引用次数: 0

摘要

在交通领域电气化的背景下,必须开发智能算法来控制电池组。智能和可重构电池是传统电池组的理想替代品,为运行和状态监测提供了新的可能性。本研究提出了一种强化学习(RL)算法,用于平衡基于拓扑结构半桥和电池模块化多级管理(BM3)的可重构电池的充电状态(SoC)。作为一种 RL 算法,摊销 Q-learning (AQL) 得到了实施,它能够控制可重构电池的大量可能配置,并将经典控制方法与机器学习方法相结合。这增强了控制过程中的安全机制。作为 AQL 的神经网络,采用了由三个隐藏层组成的前馈神经元网络(FNN)。使用 12 个单元的混合级联多电平转换器进行的实验评估表明,该方法适用于平衡 SoC 并在放电过程中保持平衡状态。评估结果表明,与传统方法相比,平衡过程要慢 20.3%。不过,AQL 在多目标优化方面显示出巨大的潜力,可作为一种 RL 算法应用于电力电子器件的控制。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Controlling Algorithm of Reconfigurable Battery for State of Charge Balancing Using Amortized Q-Learning
In the context of the electrification of the mobility sector, smart algorithms have to be developed to control battery packs. Smart and reconfigurable batteries are a promising alternative to conventional battery packs and offer new possibilities for operation and condition monitoring. This work proposes a reinforcement learning (RL) algorithm to balance the State of Charge (SoC) of reconfigurable batteries based on the topologies half-bridge and battery modular multilevel management (BM3). As an RL algorithm, Amortized Q-learning (AQL) is implemented, which enables the control of enormous numbers of possible configurations of the reconfigurable battery as well as the combination of classical controlling approaches and machine learning methods. This enhances the safety mechanisms during control. As a neural network of the AQL, a Feedforward Neuronal Network (FNN) is implemented consisting of three hidden layers. The experimental evaluation using a 12-cell hybrid cascaded multilevel converter illustrates the applicability of the method to balance the SoC and maintain the balanced state during discharge. The evaluation shows a 20.3% slower balancing process compared to a conventional approach. Nevertheless, AQL shows great potential for multiobjective optimizations and can be applied as an RL algorithm for control in power electronics.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Batteries
Batteries Energy-Energy Engineering and Power Technology
CiteScore
4.00
自引率
15.00%
发文量
217
审稿时长
7 weeks
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信