Learning symmetries and non-Euclidean data representations via collective dynamics of generalized Kuramoto oscillators

Abstract

Learning low-dimensional representations of data is the central problem of modern machine learning (ML). Recently, it has been widely recognized that some ubiquitous data sets are more faithfully represented in curved manifolds, rather than in Euclidean spaces. This observation motivated extensive experiments with non-Euclidean deep learning architectures. In many setups, data embeddings on spheres, hyperbolic spaces or matrix manifolds enable more compact models and efficient algorithms. In the present paper we argue that Kuramoto models (including their higher-dimensional generalizations) provide a powerful framework for inferring intrinsic curvature and hidden symmetries of the data. Kuramoto ensembles naturally encode actions of transformation groups (such as special orthogonal, unitary and Lorentz groups). Following decades of studies of the Kuramoto model and its extensions, the corresponding group-theoretic framework is well established. This provides a solid theoretical foundation for geometry-informed architectures. We overview families of probability distributions on spheres and hyperbolic balls which are associated with various Kuramoto models. These probability distributions provide statistical models for encoding uncertainties and designing inference algorithms in geometric deep learning. Due to their favorable properties, Kuramoto networks can be trained via standard backpropagation method.