Graph Neural Networks as Application of Distributed Algorithms

Abstract

At first sight, distributed computing and machine learning are two distant areas in computer science. However, there are many connections, for instance in the area of graphs, which are the focus of my talk. Distributed computing has studied distributed graph algorithms for many decades. Meanwhile in machine learning, graph neural networks are picking up steam. When it comes to dealing with graphical inputs, one can almost claim that graph neural networks are an application of distributed algorithms. I will introduce central concepts in learning such as underreaching and oversquashing, which have been known in the distributed computing community for decades, as local and congest models. In addition I am going to present some algorithmic insights, and a software framework that helps with explaining learning. Generally speaking, I would like to present a path to learning for those who are familiar with distributed message passing algorithms. This talk is based on a number of papers recently published at learning conferences such as ICML and NeurIPS, co-authored by Pál András Papp and Karolis Martinkus.