Kolmogorov–Arnold networks with gumbel softmax for recovering network structure and forecasting complex systems

Abstract

Reconstructing network connections and dynamics simultaneously is essential for understanding complex systems and developing appropriate strategies. Graph neural networks (GNNs) and Transformers are frequently employed to model dynamics. These models depend on MultiLayer Perceptrons (MLPs) with linear weights and predetermined activation functions to integrate information, which somewhat limits their expressive capability. Additionally, the inferred structure may be inaccurate, as the dynamics and correlations within the complex system are interdependent. As a result, its forecasting ability may be limited. To enhance fitting accuracy of both structure and dynamics, we introduce the Kolmogorov–Arnold Networks (KANs) and Gumbel-softmax technique for modeling continuous and discrete dynamics via spline-parameterized univariate functions and explicit network structure, respectively. Experiments conducted on both simulated data and a real dataset demonstrate that two novel approaches, GGNKAN and more efficient GKAN, can effectively capture the complex and nonlinear functions of the nodes and their interactions, outperforming the Gumbel Graph Networks (GGNs), the state-of-the-art method, in terms of interactions recovery and/or prediction error.