A slot-based energy storage decision-making approach for optimal Off-Grid telecommunication operator

Abstract

This paper proposes a slot-based energy storage approach for decision-making in the context of an Off-Grid telecommunication operator. We consider network systems powered by solar panels, where harvest energy is stored in a battery that can also be sold when fully charged. To reflect real-world conditions, we account for non-stationary energy arrivals and service demands that depend on the time of day, as well as the failure states of PV panel. The network operator we model faces two conflicting objectives: maintaining the operation of its infrastructure and selling (or supplying to other networks) surplus energy from fully charged batteries. To address these challenges, we developed a slot-based Markov Decision Process (MDP) model that incorporates positive rewards for energy sales, as well as penalties for energy loss and battery depletion. This slot-based MDP follows a specific structure we have previously proven to be efficient in terms of computational performance and accuracy. From this model, we derive the optimal policy that balances these conflicting objectives and maximizes the average reward function. Additionally, we present results comparing different cities and months, which the operator can consider when deploying its infrastructure to maximize rewards based on location-specific energy availability and seasonal variations. Experimental results show that our proposed algorithm outperforms classical methods in large-scale scenarios. While Relative Value Iteration algorithm remains competitive on smaller instances, its convergence time increases significantly under strict precision requirements (e.g., $ϵ<10^{-10}$). In contrast, our method maintains both speed and robustness, solving MDPs with up to $2\times 10^5$ states and 100 actions in under one hour, whereas standard approaches exceed $10^4$ seconds.