Deterministic Convergence of Backpropagation Algorithm with Cyclic Dropconnect for Linear Output Neural Networks

Abstract

In this paper, we consider the backpropagation algorithm with cyclic DropConnect (BPA-CDC) for neural networks with linear output. Under mild conditions, we establish the deterministic convergence theory for BPA-CDC, showing that the cost function tends to a constant, the gradient tends to zero, and the weight vector tends to a point. Simulation results are provided to validate our theoretical findings.