Optimizing energy efficiency in unrelated parallel machine scheduling problem through reinforcement learning

Abstract

The industrial sector plays a significant role in global energy consumption and greenhouse gas emissions. To reduce this environmental impact, it's crucial to implement energy-efficient manufacturing systems that utilize sustainable materials and optimize energy usage. This can lead to benefits such as reduced carbon footprints and cost savings.

In recent years, metaheuristic approaches have been focused on minimizing energy consumption within the Unrelated Parallel Machine Scheduling Problem (UPMSP). Traditional methods often overlook complex factors like release dates, due dates, and job setup times. This research introduces a novel algorithm that integrates reinforcement learning (RL) with a genetic algorithm (GA) to address this gap.

The proposed RLGA algorithm, rooted in the dynamic field of evolutionary reinforcement learning, breaks down policies into smaller components to isolate essential parameters for problem-solving. Through comprehensive analysis, hyperparameters that influence optimal results are identified, facilitating automated hyperparameter selection and optimization. The expert system takes into account problem characteristics such as machine or job saturation, job overlap, and the maximum values of target variables, allowing instances to be grouped into clusters. These clusters are solved using a genetic algorithm with varying combinations of mutation and crossover hyperparameters. The most suitable approach for each cluster is determined by analyzing the results, and this configuration of hyperparameters is applied iteratively to optimize the solution search.

The effectiveness of RLGA is evaluated across benchmark instances with different complexities, machine sets, jobs, and constraints. Comprehensive comparisons against existing methods highlight the superior performance and efficiency of RLGA in optimizing energy use and solution quality. Experimental results show that RLGA outperforms well-known solvers like CPO, CPLEX, OR-tools, and Gecode, making it a promising approach for optimizing energy-efficient manufacturing systems.