Multi-Agent Actor-Critic Method for Joint Duty-Cycle and Transmission Power Control

Abstract

In energy-harvesting Internet of Things (EH-IoT) wireless networks, maintaining energy neutral operation (ENO) is crucial for their perpetual operation and maintenance-free property. Guaranteeing this ENO condition and optimal power-performance trade-off under transient harvested energy and wireless channel quality is particularly challenging. This paper proposes a multi-agent actor-critic reinforcement learning for modulating both the transmitter duty-cycle and output power based on the state-of-buffer (SoB) and the state-of-charge (SoC) information as a state. Thanks to these buffers, differently from the state-of-the-art, our solution does not require any model of the wireless transceiver nor any direct measurement of both harvested energy and wireless channel quality for adapting to these uncertainties. Simulation results of a solar powered EH-IoT node using real-life outdoor solar irradiance data show that the proposed method achieves better performance without system failures throughout a year compared to the state-of-the-art that suffers some system downtime. Our approach also predicts almost no system fails during five years of operation.