Accelerating Sample-based GNN Training by Feature Caching on GPUs

Abstract

The existing graph neural network (GNN) systems adopt sample-based training on large-scale graphs over multiple GPUs. Although they support large-scale graph training, large data loading overhead is still a bottleneck. In this work, we propose SCGraph, a method that supports GPU high-speed feature caching. We classify the graph vertices sorted by out-degrees. For high out-degree vertices, we set grading caches via different GPUs to increase the overall cache content through NVLink high-speed data transmission between them. For low out-degree vertices, we expand training vertices’ neighborhood in advance to regenerate cache. We evaluate SCGraph against two state-of-the-art industrial GNN frameworks, i.e., DGL and PaGraph on two datasets Reddit and ogbn-products. Experimental results show that SCGraph achieves up to 1.83× performance speedup over the state-of-the-art baselines.