Statistical Search for Hierarchical Linear Optimal Representations of Images

Although linear representations of images are widely used in appearance-based recognition of objects, the frequently used ideas, such as PCA, ICA, and FDA, are often found to be suboptimal. A stochastic search algorithm has been proposed recently [4] for finding representations that are optimal for specific tasks and datasets. However, this search algorithm is computationally efficient only when the image size is relatively small. Here we propose a hierarchical learning algorithm to speed up the search. The proposed approach decomposes the original optimization problem into several stages according to a hierarchical organization. In particular, the following idea is applied recursively: (i) reduce the image dimension using a shrinkage matrix, (ii) optimize the recognition performance in the reduced space, and (iii)expand the optimal subspace to the bigger space in a pre-specified way. We show that the optimal performance is maintained in the last step. By applying this decomposition procedure recursively, a hierarchy of layers is formed. This speeds up the original algorithm significantly since the search is performed mainly in reduced spaces. The effectiveness of hierarchical learning is illustrated on a popular database, where the computation time is reduced by a large factor compared to the original algorithm.