L.L. Thurstone, The law of effect and the dynamics of deep learning “a deep history of deep learning”

Abstract

Deep learning (DL), a variant of the neural network algorithms originally proposed in the early twentieth century, has resulted in a renaissance of artificial intelligence. Despite the growing dominance of DL networks, little is understood about the learning mechanism that makes these networks so effective across such a wide range of applications. Drawing on a century of psychological learning theory (e.g., LL Thurstone), an account is offered of the learning mechanism that may enable DL networks to perform so successfully across so many different tasks. Specifically, evidence is provided that learning in DL networks is fit best by a hyperbolic function. This function is independent of hyper/meta parameters–not a scaling function but a learning curve to a specific equilibrium, a function that also entails an autocatalytic mechanism through which complex structures, abstracted from sensory features, can in principle create and support cognitive function in both biological and artificial systems. Keywords: Learning theory, Deep Learning, hyperbolic, autocatalytic, AI, Thurstone