From "More Is Different" to Algorithmic Emergence: Regularity, Compression, and the Limits of Discovery.

Abstract

Scientific discovery seeks regularities that support explanation and prediction. Compression makes their reuse explicit: shared structure is described once, while parameters specify individual cases. A model can, therefore, pay for itself through repeated use even when its description is not minimal, provided it captures reusable structure in the data. Because regularities are often easier to identify in simpler systems, a reductionist approach is often employed: discover the laws of the parts and treat them as fundamental. Yet knowing those laws does not by itself provide useful coarse-grained models of the larger systems they compose. Here we formulate Anderson's distinction between reduction and construction for finite algorithmic observers and prove three barriers to such construction. First, an observer's coarse-grained record may retain so much information about initial or boundary conditions that no substantially shorter description exists, even when the underlying laws are simple and known. Second, when a shorter description does exist, open-ended search can eventually find one, but there is no computable bound on how long this may take, and no algorithm that always halts with a valid description can succeed in every case. Third, every such algorithm has blind spots at all sufficiently large lengths: records it leaves unshortened even though they admit descriptions of only logarithmic length. Yet observers do discover useful models. We call the event in which an observer acquires a representation relevant to its objective and a reusable model that reveals previously unavailable regularity algorithmic emergence. We give a sufficient certificate in explicit code lengths: the pair passes when, with its own description cost included, it compresses the retained data relative to an agreed baseline and yields further savings on later observations. Even for a stream that repeats one block, no computable procedure that always halts with valid codes can guarantee finding a passing pair with the observations available whenever one exists, nor remain within a fixed number of bits of the best qualifying complete code. Favorable structure, including symmetry and restricted model classes, can nevertheless make discovery feasible.