A text-only weakly supervised learning framework for text spotting via text-to-polygon generator

Advanced text spotting methods typically rely on large-scale, meticulously labeled datasets to achieve satisfactory performance. However, annotating fine-grained positional information of texts in real-world scene images is extremely costly and time-consuming. Although some weakly supervised methods have been developed to reduce annotation costs, they face two major challenges: 1) their performance significantly lags behind the fully supervised counterparts, and 2) They are tightly coupled with specific text spotting models, meaning that switching to a different model would require retraining and incur substantial computational costs. To address these limitations, we propose a novel text-only weakly supervised learning framework for text spotting via text-to-polygon generator. In the first stage, we pretrain a text-to-polygon generator on an auxiliary dataset, e.g., synthetic or public datasets, where full annotations are readily accessible. In the second stage, given real-world target datasets annotated with text-only labels, we employ the pretrained generator to produce pseudo polygon labels, thereby constructing a pseudo-labeled supervised dataset for training text spotting models. To ensure high-quality pseudo polygon labels, the text-to-polygon generator first identifies all candidate text regions, then filters those that are relevant to the target text, and finally predicts their precise spatial locations. Notably, this generator requires only a single pretraining session and can subsequently be applied to any text spotting model and target text-only dataset without incurring additional costs. Extensive experiments on public benchmarks demonstrate that our method can significantly reduce labeling costs while maintaining competitive performance.