On the Limits of Stochastic Computing

Abstract

Stochastic computing (SC) provides large benefits in area and power consumption at the cost of limited computational accuracy. For this reason, it has been proposed for computation intensive applications that can tolerate approximate results such as neural networks (NNs) and digital filters. Most system implementations employing SC are referred to as stochastic circuits, even though they can have vastly different properties and limitations. In this work, we propose a distinction between strongly and weakly stochastic circuits, which provide different options and trade-offs for implementing SC operations. On this basis, we investigate some fundamental theoretical and practical limits of SC that have not been considered before. In particular, we analyze the limits of stochastic addition and show via the example of a convolutional NN that these limits can restrict the viability of strongly stochastic systems. We further show that theoretically all non-affine functions do not have exact SC implementations and investigate the practical implications of this discovery.