High-level visual processing in the lateral geniculate nucleus revealed using goal-driven deep learning

Background

The Lateral Geniculate Nucleus (LGN) is an essential contributor to high-level visual processing despite being an early subcortical area in the visual system. Current LGN computational models focus on its basic properties, with less emphasis on its role in high-level vision.

New method

We propose a high-level approach for encoding mouse LGN neural responses to natural scenes. This approach employs two deep neural networks (DNNs); namely VGG16 and ResNet50, as goal-driven models. We use these models as tools to better understand visual features encoded in the LGN.

Results

Early layers of the DNNs represent the best LGN models. We also demonstrate that numerosity, as a high-level visual feature, is encoded, along with other visual features, in LGN neural activity. Results demonstrate that intermediate layers are better in representing numerosity compared to early layers. Early layers are better at predicting simple visual features, while intermediate layers are better at predicting more complex features. Finally, we show that an ensemble model of an early and an intermediate layer achieves high neural prediction accuracy and numerosity representation.

Comparison with existing method(s)

Our approach emphasizes the role of analyzing the inner workings of DNNs to demonstrate the representation of a high-level feature such as numerosity in the LGN, as opposed to the common belief about the simplicity of the LGN.

Conclusions

We demonstrate that goal-driven DNNs can be used as high-level vision models of the LGN for neural prediction and as an exploration tool to better understand the role of the LGN.