BadSTR: Backdoor Attack on Scene Text Recognition in IoT

Recent researches have shown that nonsequential tasks based on deep neural networks (DNNs), such as image classification and object detection, are vulnerable to backdoor attacks, leading to incorrect model predictions. As a crucial task in computer vision, scene text recognition (STR) is widely used in Internet of Things (IoT) fields, such as intelligent transportation system and intelligent surveillance. Given its importance, ensuring the security and accuracy of STR models is critical. However, there are currently no studies on STR backdoor attacks. In this article, we make the first attempt to validate backdoor threats on STR models by using a patch-based attack method. Our experimental results confirm that STR models can be successfully compromised with attack success rate (ASR) of over 80% on most datasets. However, we also reveal a critical flaw: the patch-based attack lacks robustness due to the specific preprocessing in STR models [such as resizing and thin-plate spline (TPS) rectification], which distort or eliminate the backdoor triggers. To address this, we further propose BadSTR, a novel backdoor attack method that uses semantic text sequences as triggers. Extensive experiments on eight benchmark datasets show that our proposed BadSTR achieves ASR of over 90% for most model-dataset combinations with significantly improved robustness.