RankSearch: An Automatic Rank Search Towards Optimal Tensor Compression for Video LSTM Networks on Edge

Abstract

Various industrial and domestic applications call for optimized lightweight video LSTM network models on edge. The recent tensor-train method can transform space-time features into tensors, which can be further decomposed into low-rank network models for lightweight video analysis on edge. The rank selection of tensor is however manually performed with no optimization. This paper formulates a rank search algorithm to automatically decide tensor ranks with consideration of the trade-off between network accuracy and complexity. A fast rank search method, called RankSearch, is developed to find optimized low-rank video LSTM network models on edge. Results from experiments show that RankSearch achieves a $4.84 >$ reduction in model complexity, and $1.96\times$ speed-up in run time while delivering a 3.86% accuracy improvement compared with the manual-ranked models.