Size Adaptation of Separable Dictionary Learning with Information-Theoretic Criteria

Abstract

In sparse representation problems, the size of the dictionary is critical to the performance of the learning algorithm and, apart from loose guidelines concerning dictionary integrity, there is little indication on how to determine the optimal size. Information-theoretic criteria (ITC), used generally for model selection, have recently been employed for the task. This paper extends the work for the case of separable dictionaries, by modifying the Extended Renormalized Maximum Likelihood criterion to the 2D model and proposes an adaptation algorithm that almost entirely relies on the ITC score. Results in terms of mean size recovery rates are within 1 atom away from the true size, while representation errors are consistently below those obtained when applying dictionary learning with the known size.