A novel stepsize for gradient descent method

Abstract

We propose a novel adaptive stepsize for the gradient descent scheme to solve unconstrained nonlinear optimization problems. With the convex and smooth objective satisfying locally Lipschitz gradient we obtain the complexity $O\left(\frac{1}{k}\right)$ of $f\left(x^k\right)-f_{⁎}$ at most. By using the idea of the new stepsize, we propose another new algorithm based on the projected gradient for solving a class of nonconvex optimization problems over a closed convex set. The computational experiments show the efficiency of the new method.