To Think Like a Vertex (or Not) for Distributed Training of Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) train neural networks that combine the topological properties of a graph with the vertex and edge features to perform tasks such as node classification and link prediction. We propose a novel middleware that approaches GNN training from the perspective of a vertex-centric model (VCM) of distributed graph processing and overlays neural network training over it. Giraph Graph Neural Network (G2N2) uses a three-phase execution pattern by construction a distributed computation graph per mini-batch, and maps the forward and backward passes of the GNN training to VCM. We implement a prototype of G2N2 in Apache Giraph and report results from a preliminary evaluation using two real-world graphs on a commodity cluster.