Multilingual natural scene text detection via global feature fusion

Natural scene text detection is a significant challenge in computer vision, with tremendous potential applications in multilingual, diverse, and complex text scenarios. A multilingual text detection model based on the Cascade Mask R-CNN is proposed to address the challenges of low accuracy and high difficulty in detecting multilingual text in natural scenes. In response to the challenges posed by multilingual text images with multiple character sets and various font styles, the SFM Swin Transformer feature extraction network is introduced to increase the robustness of character and font detection across different languages. To address the considerable variation in text scales and complex arrangements in natural scene text images, the AS-HRFPN feature fusion network is constructed by incorporating an adaptive spatial feature fusion module and a spatial pyramid pooling module. The feature fusion network improvements enhance the model’s ability to detect text sizes and orientations. Furthermore, to address the complexity of high background diversity and variations in font letter morphology across different languages in multilingual natural scene text images, existing methods often need better detection performance because of the need for global information caused by limited local receptive fields. To mitigate this, a global semantic segmentation branch is introduced to extract and preserve global features to guide text detection. This study collected and constructed a real-world, multilingual, natural scene text image dataset, and comprehensive experiments and analyses were conducted. The experimental results demonstrate that the proposed algorithm achieves an F-measure of 85.02%, which is 4.71% higher than that of the baseline model. Extensive cross-dataset validations on the MSRA-TD500, ICDAR2017MLT, and ICDAR2015 datasets were also conducted to verify the generality of our approach.