Dual-Actor Critic Adaptive Energy Management Method for EH-WSN Based on Battery Energy Neutral Operation

Abstract

Energy harvesting wireless sensor nodes collect energy in a nonlinear dynamic change, resulting in low ability to dynamically match the collected and consumed energy of the node in the process of maintaining energy neutral operation (ENO).To address this problem, the concept of battery ENO (BENO) is proposed by analyzing the battery energy buffer characteristics, and the dual-actor critic energy harvesting wireless sensor node adaptive energy management (DAC) method is proposed based on BENO. The method designs a dual-actor critic structure, senses ENO through the battery energy neutral value, and dynamically adjusts the duty cycle based on this value, in order to achieve the purpose of improving the ability of dynamically matching the collected energy with the consumed energy. The experiments are carried out on three datasets with different energy harvesting capabilities, and compared and analyzed with three classical algorithms, RLman, AQL and FQL. The experimental results show that compared with the other three classical algorithms, DAC sacrifices a small amount of duty cycle, but effectively improves the stability of battery energy, and improves the energy utilization and ENO performance. The BENO concept and the DAC methodology can provide guidance and references for the research of energy management in energy-harvesting wireless sensor nodes.