Hierarchical layered multigraph network with scale importance estimation for image classification

Abstract

This work introduces a novel image representation and processing approach using Graph Neural Networks (GNNs). We propose a multigraph representation named HiErarchical Layered Multigraph (HELM), which explicitly encodes spatial and hierarchical relationships as distinct edge types, overcoming the limitations of existing methods that fail to fully exploit relational information in images. A multi-scale representation is generated through hierarchical segmentation of a superpixel base graph, enabling the computation of spatial and hierarchical relationships within and across scales. To effectively process this multi-relational information, we introduce the HiErarchical Layered Multigraph Network (HELMNet), a novel GNN architecture incorporating specialized mechanisms for selectively aggregating and fusing information from each distinct edge type. Additionally, it includes a Region Graph Readout (RGR) module that employs an attention mechanism to dynamically weight the contribution of each representation scale during the aggregation process for classification. Experimental results demonstrate the greater efficacy of HELMNet for image classification. Compared to hierarchical models that do not distinguish between edge types, HELMNet obtains substantial average accuracy gains of 2.1% (significant at 3% level) and 10% (significant at 0.1% level) on the CIFAR-10 and STL-10 datasets, respectively. On the EUROSAT dataset, HELMNet achieves over 95% accuracy, requiring only 0.73% of the best-performing state-of-the-art model size (in number of parameters). For the more demanding and high-resolution RESISC45 dataset, the proposed model still delivers impressive results, achieving an accuracy of over 85%.