GNNBoost: Accelerating sampling-based GNN training on large scale graph by optimizing data preparation

Abstract

Graph Neural Networks (GNNs) have successfully extended deep learning from traditional Euclidean spaces to complex graph structures. Sampling-based GNN training has been widely adopted for large-scale graphs without compromising accuracy. However, the graph irregularity results in imbalanced sampling workloads, making it challenging for existing GNN systems to effectively utilize GPU resources for graph sampling. Additionally, in GNN systems where both topology and feature caches are enabled, differences in characteristics and purposes of cache data complicate the allocation of GPU memory for these two caches with minimal overhead. To address these challenges, we propose GNNBoost, a framework designed to accelerate GNN training. GNNBoost consists of two key innovations. First, GNNBoost introduces a degree-oriented sampling schedule that groups training vertices based on their degrees and applies tailored sampling strategies to balance GPU workloads and improve sampling performance. Second, GNNBoost develops a low-overhead cache space allocation mechanism that accurately determines the optimal cache sizes for graph topology and features across different workloads, minimizing both space and time overheads. We conduct a comprehensive evaluation of GNNBoost through various GNN models and large graph datasets, demonstrating that it significantly outperforms existing GNN training systems.