Integrating asynchronous advantage actor–critic (A3C) and coalitional game theory algorithms for optimizing energy, carbon emissions, and reliability of scientific workflows in cloud data centers

The growth of workflow as a service (WFaaS) has become more intricate with the increasing variety and number of workflow module applications and expanding computing resources. This complexity leads to higher energy consumption in data centers, negatively impacting the environment and extending processing times. Striking a balance between reducing energy and carbon emissions and maintaining scheduling reliability is challenging. While deep reinforcement learning (DRL) approaches have shown significant success in workflow scheduling, they require extensive training time and data due to application homogeneity and sparse rewards, and they do not always guarantee effective convergence. On the other hand, experts have developed various scheduling policies that perform well for different optimization goals, but these heuristic strategies lack adaptability to environmental changes and specific workflow optimization. To address these challenges, an enhanced asynchronous advantage actor–critic (A3C) method combined with merge-and-split-based coalitional game theory is proposed. This approach effectively guides DRL learning in large-scale dynamic scheduling issues using optimal policies from the expert pool. The merge-and-split-based method prioritizes computing nodes based on their preemptive characteristics and resource heterogeneity, ensuring reliability-aware workflow scheduling that maps applications to computing resources while considering the dynamic nature of energy costs and carbon footprints. Experiments on real and synthesized workflows show that the proposed algorithm can learn high-quality scheduling policies for various workflows and optimization objectives, achieving energy efficiency improvements of 7.65% to 19.32%, carbon emission reductions of 3.13% to 14.76%, and reliability enhancements of 17.22% to 41.65%.