Duality in neurocomputational inductive inference: a simulationist perspective

Abstract

Inductive inference is the process of inferring a description of a function from a finite subset of its graph. Connectionist inductive inference typically involves gradient descent algorithms in weight space. When inferring functions of unbounded sequences such algorithms run on recurrent nets and become computationally expensive. A broader framework for inductive inference is presented, and it is shown that such problems admit a dual approach, which can be phrased in terms of the simulation-as-homomorphism perspective in systems theory. Whereas the usual approach adapts the dynamics of the net to match the dynamics of the target system, the dual approach keeps the dynamics fixed and learns a homomorphism from the net to the target. The latter technique is promising because of its efficiency and its direct applicability to learning by continuous nonconnectionist system, such as neural fields.<>