A 36 mJ/Inf Convolution Accelerator With Reduced Memory Access and Regrouped Sparse Kernels for Environment Sound Classification on Edge Devices

Abstract

Efficient environment sound classification (ESC) on edge devices is valuable for applications requiring continuous, long-term monitoring. Existing ESC processors have demonstrated great reductions in latency and resource occupation. However, model sparsity and computation flow still require further optimization. In this brief, we propose an end-to-end ultra-lightweight Depthwise Separable Convolution (DSC) neural network, E2E-ULDSC-Pruned, which is made publicly available as an open-source release. To implement this model, a customized accelerator featuring pipelined DSC computation and regrouped sparse kernels is developed, achieving 36mJ/Inference in ZCU102 FPGA (254ms latency and 143mW power consumption), which is superior to recent works.