Complex-valued soft-log threshold reweighting for sparsity of complex-valued convolutional neural networks

Complex-valued convolutional neural networks (CVCNNs) have been demonstrated effectiveness in classifying complex signals and synthetic aperture radar (SAR) images. However, due to the introduction of complex-valued parameters, CVCNNs tend to become redundant with heavy floating-point operations. Model sparsity is emerged as an efficient method of removing the redundancy without much loss of performance. Currently, there are few studies on the sparsity problem of CVCNNs. Therefore, a complex-valued soft-log threshold reweighting (CV-SLTR) algorithm is proposed for the design of sparse CVCNN to reduce the number of weight parameters and simplify the structure of CVCNN. On one hand, considering the difference between complex and real numbers, we redefine and derive the complex-valued log-sum threshold method. On the other hand, by considering the distinctive characteristics of complex-valued convolutional (CConv) layers and complex-valued fully connected (CFC) layers of CVCNNs, the complex-valued soft and log-sum threshold methods are respectively developed to prune the weights of different layers during the forward propagation, and the sparsity thresholds are optimized during the backward propagation by inducing a sparsity budget. Furthermore, different optimizers can be integrated with CV-SLTR. When stochastic gradient descent (SGD) is used, the convergence of CV-SLTR is proved if Lipschitzian continuity is satisfied. Experiments on the RadioML 2016.10A and S1SLC-CVDL datasets show that the proposed algorithm is efficient for the sparsity of CVCNNs. It is worth noting that the proposed algorithm has fast sparsity speed while maintaining high classification accuracy. These demonstrate the feasibility and potential of the CV-SLTR algorithm.