Novel Siamese CNN-Transformer Architecture for Efficient and Robust Cross-Domain Image Recognition

Abstract

Machine learning models often struggle with limited training data, particularly in multi-domain scenarios where stylistic variations and data scarcity pose challenges. Although Siamese neural networks have shown promise in resource-limited settings, their generalization ability across various image processing domains, especially for culturally specific scripts (e.g., Persian/Arabic numerals), remains understudied. This study introduces a Siamese-CNN with a multi-scale dynamic self-attention module, designed to enhance feature extraction and cross-domain generalization across various handwritten digit (Persian/Arabic, English), and fashion item datasets. The proposed framework integrates five distinct Siamese-CNN architectures, incorporating multi-scale attention mechanisms and dynamic weight-sharing strategies to improve feature discrimination in low-data regimes. The attention module refines feature representations adaptively, mitigating overfitting while capturing fine-grained details across different styles and scripts. Extensive experiments on the Hoda dataset, a benchmark with complex stroke structures and cultural specificity, demonstrate the superiority of the proposed model, achieving 98.34% accuracy a 6.01% improvement over previous Siamese approaches while maintaining high precision (99.32%), recall (97.35%), and F1-score (98.32%). With an average epoch time of 16.26 s, the model is computationally efficient, making it suitable for real-time applications. This study advances Siamese-CNNs for cross-domain image recognition by: (1) introducing a multi-scale adaptive attention strategy for feature refinement, (2) validating generalization across eight datasets, and (3) demonstrating the role of dynamic weight-sharing in improving representation learning under constrained resources. The proposed framework outperforms conventional methods in both accuracy and efficiency, paving the way for enhanced image recognition in handwritten text digitization, multilingual document analysis, and real-time visual perception tasks.