Adaptive Dynamic Programming Based Motion Control of Autonomous Underwater Vehicles

In this paper, Adaptive Dynamic Programming (ADP) technique is utilized to achieve optimal motion control of Autonomous Underwater Vehicle (AUV) System. The paper proposes a model-free based method that takes into consideration the actuator input and obstacle position while tracing an optimal path. The concept of machine learning enables to develop a path-planner which aims to avoid collisions with static obstacles. The ADP approach is realized to approximate the solution of the cost functional for optimization purpose by which the positions of the locally situated obstacles need not be priori-known until they are within a designed approximation safety envelope. The methodology is implemented to achieve the path-planning objective using dynamic programming technique. The Least-squares policy method serves as a recursive algorithm to approximate the value function for the domain, providing an approach for the finite space discrete control system. The concept behind the design of an obstacle-free path finder is to generate an optimal action that minimizes the local cost, defined by a functional, under constrained optimization. The most advantageous value function is described by the Hamilton Jacobi Bellman (HJB) equation, that is impractical to solve using analytical methods. To overcome the complex calculations subject to HJB, a method based on Reinforcement Learning (RL), called ADP is implemented. This paper outlines the concept of machine learning to realize a real time obstacle avoidance system.