DCBGCN: An Algorithm with High Memory and Computational Efficiency for Training Deep Graph Convolutional Network

Abstract

Graph convolutional network (GCN) has recently become a popular major focus of network representation learning (NRL). However, training a deep GCN is still quite challenging. Stacking more layers in GCN suffers vanishing gradients and GPU memory limitation and significant computational overhead. Vanishing gradients causes over-smoothing, which leads to node embedding converging to the same value. Node dependence leads to requirement to keep all the embedding in GPU memory. Neighbourhood expansion problem across GCN layers leads to significant computational overhead. In order to solve these issues, we present a model named DCBGCN (Deep and Cluster Boosting Graph Convolutional Network), which firstly uses MEITS to partition the whole graph into sub-graphs, then secondly adapts residual/dense connections between GCN layers. Extensive experiment results on PPI and Reddit tell the truth that our model can go deep with 56-layer GCN and has strong advantages in improving memory and computational efficiency. Meanwhile, we achieve promising test F1 score results on PPI and Reddit.