DyGraph

Abstract

Dynamic graph processing, execution on vertex-edge graphs that change over time, is quickly becoming a key computing need of the twenty-first century. Dynamic graph algorithms unlock real-time optimization solutions and a wide range of data-mining applications in logistics, finance, marketing, healthcare, and social media, among many others. However, graph algorithms are extremely memory-bound (i.e., their performance is limited by the bandwidth of memory accesses on the underlying hardware platform, rather than the compute capacity). Moreover, dynamic graph algorithms are being applied to increasingly-large datasets, further straining the memory systems and reducing performance. As a result, additional research is needed to leverage new memory technologies for faster, more efficient, dynamic graph-based processing. Such research is difficult without access to hitherto unavailable industrial-scale dynamic graph datasets to evaluate solutions. In this work, we present DyGraph, a dynamic graph synthetic dataset generator paired with a collection of real-world graphs in the domains of social media, recommendation systems, and fintech. We demonstrate the breadth of graph features represented in this repository and evaluate the DyGraph Generator's ability to generate synthetic graphs that mimic these real datasets. In our case study, we find that the degree distribution of DyGraph Generator datasets correlate 3 to 5.5 times more closely to real-world datasets than Power Law models, paving the way for much-needed research for high-performance dynamic graph processing.