Uncertainty through Sampling: The Correspondence of Monte Carlo Dropout and Spiking in Artificial Neural Networks

Abstract

Any organism that senses its environment only has an incomplete and noisy perspective on the world, which creates a necessity for nervous systems to represent uncertainty. While the principles of encoding uncertainty in biological neural ensembles are still under investigation, deep learning became a popular and effective machine learning method. In these models, sampling through dropout has been proposed as a mechanism to encode uncertainty. Moreover, dropout has previously been linked to variability in spiking networks under specific assumptions. We compare the relationship between dropout and spiking neuron models by means of the variation ratio over their output. We demonstrate that in cases of incomplete world knowledge (epistemic uncertainty) as well as for noisy observations (aleatoric uncertainty) both neuron models show similar uncertainty representations. These findings provide evidence that sampling could play a fundamental role in representing uncertainties in neural systems.