Robust global optimization on smooth compact manifolds via hybrid gradient-free dynamics

Abstract

It is well known that smooth autonomous dynamical systems modeled by ordinary differential equations (ODEs) cannot robustly and globally stabilize a point on compact, boundaryless manifolds. This obstruction, which is topological in nature, has significant implications for optimization problems, rendering traditional continuous-time algorithms incapable of robustly solving global optimization problems in such spaces. In turn, gradient-free optimization algorithms, which usually inherit their stability and convergence properties from their gradient-based counterparts, can also suffer from similar topological obstructions. For instance, this is the case in zeroth-order methods and perturbation-based techniques, where gradients and Hessian matrices are usually estimated in real-time via measurements or evaluations of the cost function. To address this problem, in this paper we introduce a novel class of hybrid gradient-free optimization dynamics that combine continuous-time and discrete-time feedback to overcome the obstructions that emerge in traditional ODE-based optimization algorithms evolving on smooth compact manifolds. The proposed hybrid dynamics switch between different gradient-free feedback-laws obtained by applying suitable exploratory geodesic dithers to a family of synergistic diffeomorphisms adapted to the cost function that defines the optimization problem. The use of geodesic dithers enables a suitable exploration of the manifold while simultaneously preserving its forward invariance, a property that is fundamental for many practical applications with physics-based constraints. The hybrid dynamics exploit the information obtained from the geodesic dithers to achieve robust global practical stability of the set of minimizers of the cost function. This stabilization is achieved without having direct access to the gradients of the cost functions, but rather using only real-time and continuous evaluations of the cost. Examples and numerical results are presented to illustrate the main ideas and advantages of the method.