PrediRep: Modeling hierarchical predictive coding with an unsupervised deep learning network

Abstract

Hierarchical predictive coding (hPC) provides a compelling framework for understanding how the cortex predicts future sensory inputs by minimizing prediction errors through an internal generative model of the external world. Existing deep learning models inspired by hPC incorporate architectural choices that deviate from core hPC principles, potentially limiting their utility for neuroscientific investigations. We introduce PrediRep (Predicting Representations), a novel deep learning network that adheres more closely to architectural principles of hPC. We validate PrediRep by comparing its functional alignment with hPC to that of existing models after being trained on a next-frame prediction task. Our findings demonstrate that PrediRep, particularly when trained with an all-level loss function (PrediRepAll), exhibits high functional alignment with hPC. In contrast to other contemporary deep learning networks inspired by hPC, it consistently processes input-relevant information at higher hierarchical levels and maintains active representations and accurate predictions across all hierarchical levels. Although PrediRep was designed primarily to serve as a model suitable for neuroscientific research rather than to optimize performance, it nevertheless achieves competitive performance in next-frame prediction while utilizing significantly fewer trainable parameters than alternative models. Our results underscore that even minor architectural deviations from neuroscientific theories like hPC can lead to significant functional discrepancies. By faithfully adhering to hPC principles, PrediRep provides a more accurate tool for in silico exploration of cortical phenomena. PrediRep’s lightweight and biologically plausible design makes it well-suited for future studies aiming to investigate the neural underpinnings of predictive coding and to derive empirically testable predictions.