Neuro-dynamic programming based on self-organized patterns

Abstract

This paper introduces a real-time learning control mechanism, as a robust and efficient scheme of neuro-dynamic programming. The objective of the learning controller is to optimize a certain performance measure by learning to create appropriate control actions through interacting with the environment. The controller is set out to learn to perform better over time starting with no prior knowledge about the system. The system under consideration does not render a complete system model describing its behaviors. Instead, real-time sampled measurements are available to the designer. The state measurements are first analyzed by similarity and organized by proximity. Control actions are then generated in relevance to the state patterns. A critic network serves the purpose of 'monitoring' the performance of the controller to achieve a given optimality. We provide detailed implementation, and performance evaluations of this learning controller in a cart-pole balancing problem.