DJAYA-RL: Discrete JAYA algorithm integrating reinforcement learning for the discounted {0-1} knapsack problem

Abstract

The JAYA algorithm is a swarm heuristic algorithm designed for solving continuous space problems. To apply it to the Discounted {0-1} Knapsack Problem (D{0-1}KP), it must be optimized into a discrete problem solving algorithm. Based on three decision vector encoding schemes for the D{0-1}KP, this paper discretely improves the JAYA algorithm using Q-learning and proposes three Discrete JAYA-RL (DJAYA-RL) algorithms: FirBJAYA-RL (the First Binary JAYA Algorithm Integrated with Reinforcement Learning), SimBJAYA-RL (the Simplified Binary JAYA Algorithm Integrated with Reinforcement Learning), and QJAYA-RL (the Quaternary JAYA Algorithm Integrated with Reinforcement Learning). Subsequently, a comparative analysis of the algorithm performance among the three DJAYA-RLs is conducted.

The DJAYA-RL algorithms utilize the Q-learning mechanism to adaptively control the number of grouped coding bits during individual updates. This enables the algorithms to explore the solution space in different regions more effectively. Meanwhile, the DJAYA-RL algorithms generate populations in different episodes by leveraging the information entropy of the historically optimal individual. This approach enhances the population diversity and helps avoid premature convergence.

Experimental results on the standard dataset of the D{0-1}KP demonstrate that the average solution errors of FirBJAYA-RL, SimBJAYA-RL, and QJAYA-RL are 1.1%, 0.15%, and 0.20% respectively. These results indicate that differences in encoding schemes have an impact on algorithm performance. Among the three encoding types, SimBJAYA-RL exhibits the best solution quality, while QJAYA-RL shows the best time performance. When compared with the genetic algorithm, firefly algorithm, and particle swarm algorithm for solving the D{0-1}KP, the average solution error rate of DJAYA-RL is significantly lower than that of the three swarm heuristic algorithms with corresponding encoding schemes. Moreover, compared with the three previously proposed discrete JAYA algorithms, the average solution error rate of the DJAYA-RL algorithm is significantly lower than that of the BJaya-JS, IBJA, and JayaX algorithms with corresponding encoding schemes.