Convolutional Neural Networks

5 The convolution layer 13 5.1 What is a convolution? . . . . . . . . . . . . . . . . . . . . . . . . 13 5.2 Why to convolve? . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.3 Convolution as matrix product . . . . . . . . . . . . . . . . . . . 17 5.4 The Kronecker product . . . . . . . . . . . . . . . . . . . . . . . 19 5.5 Backward propagation: updating the parameters . . . . . . . . . 20 5.6 Even higher dimensional indicator matrices . . . . . . . . . . . . 22 5.7 Backward propagation: preparing the supervision signal for the previous layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.8 Fully connected layer as a convolution layer . . . . . . . . . . . . 25