A scalable synergy-first backbone decomposition of higher-order structures in complex systems

Abstract

In the last decade, there has been an explosion of interest in the field of multivariate information theory and the study of emergent, higher-order interactions. These “synergistic” dependencies reflect information that is in the “whole” but not any of the “parts.” Arguably the most successful framework for exploring synergies is the partial information decomposition (PID). Despite its considerable power, the PID has a number of limitations that restrict its general applicability. Subsequently, other heuristic measures, such as the O-information, have been introduced, although these measures typically only provide a summary statistic of redundancy/synergy dominance, rather than direct insight into the synergy itself. To address this issue, we present an alternative decomposition that is synergy-first, scales much more gracefully than the PID, and has a straightforward interpretation. We define synergy as that information encoded in the joint state of a set of elements that would be lost following the minimally invasive perturbation on any single element. By generalizing this idea to sets of elements, we construct a totally ordered “backbone” of partial synergy atoms that sweeps the system’s scale. This approach applies to the entropy, the Kullback-Leibler divergence, and by extension, to the total correlation and the single-target mutual information (thus recovering a “backbone” PID). Finally, we show that this approach can be used to decompose higher-order interactions beyond information theory by showing how synergistic combinations of edges in a graph support global integration via communicability. We conclude by discussing how this perspective on synergistic structure can deepen our understanding of part-whole relationships in complex systems.