I2CU: A Dedicated Im2col Hardware Unit

Abstract

For Convolution Neural Network (CNN), the convolution operation for feature map and weight map usually implemented by im2col + GEMM method. However, for conventional method need expand feature map to a large feature matrix during a single kernel function based on convolution parameters (i.e. filter size, padding, and stride), then multiplication for matrixes took place in another function. Thus the conventional method will generate tons data transfer and the large feature matrix requires enormous storage space, it is hardware unfriendly.We design a hardware unit, I2CU (Im2Col Unit), a dedicated hardware unit to implement im2col in hardware friendly way. I2CU dynamically expand loaded 4D-Block return from texture unit and write back destination matrix to shared memory. I2CU can decrease the feature matrix storage space and implement im2col + GEMM in one kernel function.