A Novel Manifold Optimization Algorithm With the Dual Function and a Fuzzy Valuation Step

Abstract

Fuzzy mathematical theory is widely used, fuzzy optimization is a branch of fuzzy mathematical theory, the significant application area is artificial intelligence in computer science, especially machine learning (deep learning) and pattern recognition. Fuzzy mathematics, especially fuzzy optimization, has become a bridge between the manifold optimization theory and deep learning applications, which is an essential theoretical foundation. The manifold optimization algorithm employs the projection method, which is unstable. In order to resolve the problem, in this article, the theory and methodology of manifold optimization concerning real and complex spaces is fully considered. Our primary focus is on the Riemannian manifold, where a groundbreaking optimization algorithm with the dual function and a fuzzy valuation step is proposed. To accelerate the convergence and enhance the stability of the optimization algorithm, a novel learning rate is present, which is referred as bivariate gradual learning rate warm-up. A comprehensive analysis of its convergence rates is conducted in various scenarios and the experiments results substantiate our discoveries, and demonstrate the correctness and effectiveness of our devised algorithm.