Linearized Tensor Low-Rank Convolutional Network

Abstract

To balance deep learning performance and interpretability, convolutional sparse coding (CSC) has successfully incorporated differentiable optimization layers in place of the standard convolutional layers of traditional deep neural networks (DNN). However, CSC models prefer to fall into local sparsity under the $\boldsymbol{\ell}_\mathbf{1}$-norm, whose optimization shows limitations in preserving the geometric structure and global sparsity of tensor data. In this article, we propose a linearized tensor low-rank convolutional network (LTLRCNet) to refresh the differentiable optimization layers for DNN [e.g., Residual Network (ResNet) and U-shaped Network (UNet)], which instead optimizes the tensor low-rank minimization problem for global sparsity without ruining the intrinsic structure of data. Specifically, instead of simply replacing the $\boldsymbol{\ell}_\mathbf{1}$-norm with a tensor low-rank regularizer, we first propose a linearized tensor low-rank representation (LTLRR) model optimized by a linearized alternating direction method to decouple the dictionary and code coefficient without introducing an auxiliary variable, whose convergence is proved theoretically. Subsequently, we extend the LTLRR model into a deep unfolding network, such as the LTLRCNet, by engineering a convolutional dictionary that functions as a differentiable optimization layer, effectively substituting the conventional convolutional layer. To match the deep unfolding framework and avoid the gradient explosion, we apply a rank-one tactic bearing the Frank-Wolfe algorithm to solve the tensor low-rank minimization problem instead of conducting the cumbersome full-rank tensor singular value decomposition. Experimental results on image classification and segmentation show the potential of our LTLRCNet with its modeling-based layers to maintain significant performance and interpretability.