A comparative analysis of PPO and SAC algorithms for energy optimization with country-level energy consumption insights

Abstract

Background:

This study addresses national-scale energy optimization using deep reinforcement learning. Unlike prior works that rely on simulated environments or synthetic datasets, this research integrates real-world energy indicators, including electricity generation, greenhouse gas emissions, renewable energy share, fossil fuel dependency, and oil consumption. These indicators, sourced from the World Energy Consumption dataset, capture both developed and developing energy systems, enabling the evaluation of intelligent control policies across diverse contexts.

Methodology:

Two advanced algorithms, Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC), were implemented and trained using PyTorch across multi-phase evaluation runs (300–3000 episodes). Comparative performance analysis was conducted on key metrics: execution speed, action consistency, and reward optimization. A secondary regional analysis focused on contrasting the Balkan and Nordic countries to evaluate algorithm adaptability between highly developed and developing energy infrastructures.

Significant findings:

SAC demonstrated superior computational throughput and policy stability, making it suitable for real-time and resource-constrained environments. PPO exhibited stronger action magnitudes, enabling more assertive control signals for high-impact interventions. Both agents significantly outperformed a rule-based baseline in responsiveness and adaptability. The proposed framework represents a novel contribution by combining deep reinforcement learning with interpretable, country-level energy indicators. Future work will extend the evaluation to additional continents, including Asia, Africa, and South America, to assess global scalability and applicability.