Model-Free Neuro-Fuzzy Q-Learning Control With Swarm Intelligence

Abstract

In this article, a novel neuro-fuzzy-based evolution-guided Q-learning (EGQL) algorithm is established for solving the optimal control problem of unknown nonlinear systems. To enhance the accuracy for approximating the Q-function, the adaptive neuro-fuzzy inference system (ANFIS) is leveraged, which offers superior precision compared to traditional polynomial approximations commonly used in adaptive dynamic programming. Despite its advantages, the ANFIS-based approximation faces challenges in obtaining the derivative of the Q-function with respect to the control input. To address this limitation, evolutionary algorithms are integrated into EGQL, eliminating the need for gradient information by directly minimizing the Q-function values to derive optimal control strategies. This integration enables precise and robust exploration of the solution space, resulting in accurate and reliable control policies. Furthermore, convergence and monotonic improvement are ensured by the EGQL algorithm, making it suitable for uncertain and nonlinear environments. The effectiveness and superiority of the ANFIS-based EGQL algorithm are validated through simulation results. The developed algorithm achieves a 3.1% reduction in total cost compared to the traditional approach, demonstrating superior control performance.