A Collaborative Neural Network With Adaptive Gradient Dynamics for Nonconvex Optimization

Abstract

In this article, a collaborative neural network (CONN) with adaptive gradient dynamics is presented for addressing nonconvex optimization problems. Existing neural dynamics models often suffer from either prohibitive computational costs or unsatisfactory performance, which limits their practical effectiveness. To overcome this, an adaptive gradient neural dynamics (AGND) model is developed for local optimization in nonconvex problems, achieving reduced computational complexity and superior performance. Subsequently, a CONN is proposed that combines multiple AGND models for scattered searches with a metaheuristic rule that reinitializes neuronal states upon local convergence. Theoretical analysis establishes that the AGND model ensures local optimality, and further demonstrates the global convergence of the proposed CONN. Extensive experiments on numerous classical time-invariant nonconvex optimization benchmark problems illustrate the effectiveness and stability of the CONN. Moreover, the proposed method consistently achieves strong performance across diverse application problems, including both time-varying and time-invariant scenarios, such as robotic manipulator trajectory tracking, portfolio selection, and pressure vessel design.