Sketch-SparseNet: Sparse convolution framework for sketch recognition

Abstract

In free-hand sketch recognition, state-of-the-art methods often struggle to extract spatial features from sketches with sparse distributions, which are characterized by significant blank regions devoid of informative content. To address this challenge, we introduce a novel framework for sketch recognition, termed Sketch-SparseNet. This framework incorporates an advanced convolutional component: the Sketch-Driven Dilated Deformable Block ($S{D}^{3}B$). This component excels at extracting spatial features and accurately recognizing free-hand sketches with sparse distributions. The $S{D}^{3}B$ component innovatively bridges gaps in the blank areas of sketches by establishing spatial relationships among disconnected stroke points through adaptive reshaping of convolution kernels. These kernels are deformable, dilatable, and dynamically positioned relative to the sketch strokes, ensuring the preservation of spatial information from sketch points. Consequently, Sketch-SparseNet extracts a more accurate and compact representation of spatial features, enhancing sketch recognition performance. Additionally, we introduce the SmoothAlign loss function, which minimizes the disparity between the output features of parallel $S{D}^{3}B$ and CNNs, facilitating effective feature fusion. Extensive evaluations on the QuickDraw-414k and TU-Berlin datasets highlight our method’s state-of-the-art performance, achieving accuracies of 79.52% and 85.78%, respectively. To our knowledge, this work represents the first application of a sparse convolution framework that substantially alleviates the adverse effects of sparse sketch points. The codes are available at https://github.com/kingbackyang/Sketch-SparseNet.