Fine-grained Automatic Augmentation for handwritten character recognition

Abstract

With the advancement of deep learning-based character recognition models, the training data size has become a crucial factor in improving the performance of handwritten text recognition. For languages with low-resource handwriting samples, data augmentation methods can effectively scale up the data size and improve the performance of handwriting recognition models. However, existing data augmentation methods for handwritten text face two limitations: (1) Methods based on global spatial transformations typically augment the training data by transforming each word sample as a whole but ignore the potential to generate fine-grained transformation from local word areas, limiting the diversity of the generated samples; (2) It is challenging to adaptively choose a reasonable augmentation parameter when applying these methods to different language datasets. To address these issues, this paper proposes Fine-grained Automatic Augmentation (FgAA) for handwritten character recognition. Specifically, FgAA views each word sample as composed of multiple strokes and achieves data augmentation by performing fine-grained transformations on the strokes. Each word is automatically segmented into various strokes, and each stroke is fitted with a Bézier curve. On such a basis, we define the augmentation policy related to the fine-grained transformation and use Bayesian optimization to select the optimal augmentation policy automatically, thereby achieving the automatic augmentation of handwriting samples. Experiments on seven handwriting datasets of different languages demonstrate that FgAA achieves the best augmentation effect for handwritten character recognition. Our code is available at https://github.com/IMU-MachineLearningSXD/Fine-grained-Automatic-Augmentation