Q-learning-based hyper-heuristic algorithm for open dimension irregular packing problems

Abstract

Heuristic methods provide a computationally efficient framework for addressing two-dimensional irregular packing problems, particularly in resource-constrained industrial settings. As a typical combinatorial optimization problem, irregular packing exhibits exponential growth in computational complexity with increasing workpiece counts, while the solution space dynamically reconfigures due to geometric variability among workpieces. Although heuristic algorithms can generate feasible layouts within acceptable timeframes, their reliance on fixed search rule limits adaptability to diverse scenarios, necessitating more flexible approaches. In this paper, a hyper-heuristic algorithm based on Q-Learning is proposed to solve open dimension packing problems, including one-open and two-open dimension problems. Q-Learning is adopted as the high-level strategy for its ability to optimize low-level heuristic selection through reward-driven experience accumulation. The method incorporates a mixed encoding method for solution representation, four specialized low-level heuristic operators, a linear population decline mechanism, and an elite preservation strategy to balance exploration–exploitation. The Q-Learning controller dynamically selects operators by updating the Q-table based on Bellman’s equation. The proposed algorithm is compared to some advanced algorithms in general datasets. The results show that our method has better performance and applicability.