Convolutional and Recurrent Neural Networks

Convolutional neural networks (CNNs) are biologically-inspired variants of multi-layer perceptrons (MLPs). In biology, a visual cortex contains a complex arrangement of cells. These cells are sensitive to small subregions of the visual field. Inspired by the structure of visual cortices and cells, the notion of receptive fields and local filters are introduced as a core component of convolutional neural networks. Furthermore, in biology, the visual sub-regions are tiled to cover the entire visual field. Borrowing this idea, convolutional neural networks learn hierarchical representations by tiling and stacking multiple layers of convolutional units, which enables exploiting the strong spatially local correlation present in natural images.