Ensemble Graph Attention Networks

Graph neural networks have demonstrated its success in many applications on graph-structured data. Many efforts have been devoted to elaborating new network architectures and learning algorithms over the past decade. The exploration of applying ensemble learning techniques to enhance existing graph algorithms have been overlooked. In this work, we propose a simple generic bagging-based ensemble learning strategy which is applicable to any backbone graph models. We then propose two ensemble graph neural network models – Ensemble-GAT and Ensemble-HetGAT by applying the ensemble strategy to the graph attention network (GAT), and a heterogeneous graph attention network (HetGAT). We demonstrate the effectiveness of the proposed ensemble strategy on GAT and HetGAT through comprehensive experiments with four real-world homogeneous graph datasets and three real-world heterogeneous graph datasets on node classification tasks. The proposed Ensemble-GAT and Ensemble-HetGAT outperform the state-of-the-art graph neural network and heterogeneous graph neural network models on most of the benchmark datasets. The proposed ensemble strategy also alleviates the over-smoothing problem in GAT and HetGAT.